The following have been reloaded with a version change: 1) GCCcore/.14.3.0 => GCCcore/14.3.0 Lmod is automatically replacing "GCC/14.3.0" with "nvidia-compilers/25.9-CUDA-13". Deactivating conda environment: /e/scratch/jureap59/feuer1/miniforge3/envs/otagent Activating RL environment: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl Python executable: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python Python path check: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python [ray] RAY_TMPDIR=/tmp/ray/ray_632525 [triton_cache] Triton cache: /tmp/triton_cache_feuer1_632525 [triton_cache] TorchInductor cache: /tmp/torchinductor_cache_feuer1_632525 [proxy] ✓ Found proxychains binary at /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 [proxy] Setting up SSH tunnel to jpbl-s01-01 [proxy] SSH key: /e/home/jusers/feuer1/jupiter/.ssh/authorized_keys/id_ed25519_jsc [proxy] Tunnel port: 7003 [proxy] Node IP: 10.128.16.35 (workers will connect here) [proxy] ✓ SSH tunnel started successfully [proxy] ✓ Generated proxychains config at /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_632525.conf [proxy] - Internal traffic (10.x.x.x, 172.x.x.x, 169.254.x.x) → DIRECT [proxy] - External traffic (internet) → PROXY via tunnel [proxy] ✓ Daytona timeout settings configured [proxy] Testing proxy connectivity... [proxychains] config file found: /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_632525.conf [proxychains] preloading /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/lib/libproxychains4.so [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [proxy] ✓ Proxy connectivity test passed (huggingface.co reachable via wrapped binary) [proxy] ⚠ Tunnel not accessible at 10.128.16.35:7003 (workers may fail) [proxy] ✓ Proxy setup complete (using wrapped binary for Ray workers) [container_runtime] Using cloud backend: daytona (no local container setup) === Universal RL Training Runner === Config: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/configs/ablation-pymethods2test-seqnorm_rl_config.json Working directory: /e/scratch/jureap59/feuer1/OpenThoughts-Agent Python: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python Python version: Python 3.12.12 UV_USE_IO_URING: 0 Proxy: DISABLED (direct internet or not configured) ======================================== === RLJobRunner: ablation-pymethods2test-seqnorm === [wandb_utils] Fixing permissions on: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/wandb [wandb_utils] WandB directory ready: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/wandb HF_TOKEN=****pDbg HF_HUB_CACHE=/e/data1/datasets/playground/ot-baf/hf_hub SUPABASE_URL=https://rpzmyuapoqilpghynmza.s... (direct Supabase config) Environment configured: TENSOR_PARALLEL_SIZE=1 NUM_INFERENCE_ENGINES=56 POLICY_NUM_NODES=14 WANDB_DIR=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/wandb Starting Ray cluster with 14 nodes, 4 GPUs/node Cleaning up existing Ray instances... === Starting Ray Cluster === Nodes: 14 GPUs per node: 4 CPUs per node: 288 Head node: jpbo-001-35 (10.128.16.35) Ray port: 6379 ============================ Starting Ray head on jpbo-001-35 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_head_jpbo-001-35.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.35 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-35 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --head --node-ip-address=10.128.16.35 --port=6379 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray head on jpbo-001-35 Starting Ray worker on jpbo-001-36 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-36.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.36 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-36 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.36 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 1 on jpbo-001-36 Starting Ray worker on jpbo-001-37 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-37.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.37 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-37 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.37 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 2 on jpbo-001-37 Starting Ray worker on jpbo-001-38 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-38.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.38 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-38 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.38 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 3 on jpbo-001-38 Starting Ray worker on jpbo-001-41 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-41.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.41 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-41 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.41 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 4 on jpbo-001-41 Starting Ray worker on jpbo-001-42 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-42.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.42 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-42 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.42 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 5 on jpbo-001-42 Starting Ray worker on jpbo-001-43 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-43.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.43 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-43 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.43 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 6 on jpbo-001-43 Starting Ray worker on jpbo-001-44 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-44.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.44 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-44 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.44 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 7 on jpbo-001-44 Starting Ray worker on jpbo-001-45 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-45.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.45 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-45 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.45 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 8 on jpbo-001-45 Starting Ray worker on jpbo-001-46 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-46.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.46 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-46 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.46 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 9 on jpbo-001-46 Starting Ray worker on jpbo-001-47 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-47.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.47 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-47 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.47 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 10 on jpbo-001-47 Starting Ray worker on jpbo-001-48 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-001-48.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.48 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-001-48 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.48 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 11 on jpbo-001-48 Starting Ray worker on jpbo-004-41 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-004-41.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.185 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-004-41 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.185 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 12 on jpbo-004-41 Starting Ray worker on jpbo-004-43 (logging to /e/data1/datasets/playground/ot-baf/experiments/_ray_logs/ray_worker_jpbo-004-43.log)... Command: srun --export=ALL,VLLM_HOST_IP=10.128.16.187 --nodes=1 --ntasks=1 --gres=gpu:4 --gpu-bind=none --overlap --cpu-bind=none -w jpbo-004-43 bash -c /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f "$PROXYCHAINS_CONF_FILE" ray start --address=10.128.16.35:6379 --node-ip-address=10.128.16.187 --num-gpus=4 --num-cpus=288 --block --object-store-memory=42949672960 Started Ray worker 13 on jpbo-004-43 Waiting for cluster (56 GPUs, 14 nodes)... Connecting to Ray at 10.128.16.35:6379 (expecting 14 nodes, 56.0 GPUs) Ray connection established, polling for resources... [Ray wait] nodes=14/14 GPUs=56.0/56.0 resources={'memory': 10305948286976.0, 'node:10.128.16.48': 1.0, 'CPU': 4032.0, 'object_store_memory': 601295421440.0, 'accelerator_type:GH200': 14.0, 'GPU': 56.0, 'node:10.128.16.42': 1.0, 'node:10.128.16.47': 1.0, 'node:10.128.16.187': 1.0, 'node:10.128.16.44': 1.0, 'node:10.128.16.36': 1.0, 'node:10.128.16.38': 1.0, 'node:10.128.16.35': 1.0, 'node:__internal_head__': 1.0, 'node:10.128.16.43': 1.0, 'node:10.128.16.37': 1.0, 'node:10.128.16.45': 1.0, 'node:10.128.16.46': 1.0, 'node:10.128.16.185': 1.0, 'node:10.128.16.41': 1.0} ✓ Ray cluster ready === Ray Cluster Ready === Address: 10.128.16.35:6379 Total GPUs: 56 ========================= Ray cluster ready at 10.128.16.35:6379 Total GPUs available: 56 [RLJobRunner] Pinggy check: url=False, token=False, needs_tunnel=False (agent=terminus-2, env=daytona) [RLJobRunner] No Pinggy tunnel needed, using local vLLM Running SkyRL: Python: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python Entrypoint: examples.terminal_bench.entrypoints.main_tbench Args: 120 Hydra arguments Working dir: /e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train Using proxychains binary: /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 Executing command with srun: /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/bin/proxychains4 -f $PROXYCHAINS_CONF_FILE /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/bin/python -m examples.terminal_bench.entrypoints.main_tbench +terminal_bench_config=terminal_bench trainer.strategy=fsdp2 trainer.algorithm.advantage_estimator=rloo_n trainer.algorithm.use_kl_loss=false trainer.algorithm.kl_loss_coef=0.0 trainer.algorithm.eps_clip_low=0.2 trainer.algorithm.eps_clip_high=0.05 trainer.algorithm.loss_reduction=seq_mean_token_sum_norm_global trainer.epochs=2 trainer.max_steps=80 trainer.update_epochs_per_batch=1 trainer.train_batch_size=64 trainer.policy_mini_batch_size=64 trainer.eval_batch_size=64 trainer.micro_forward_batch_size_per_gpu=4 trainer.micro_train_batch_size_per_gpu=1 trainer.max_prompt_length=999999 trainer.eval_interval=999999 trainer.eval_before_train=false trainer.ckpt_interval=2 trainer.resume_mode=latest trainer.hf_save_interval=5 ++trainer.hf_hub_repo_id=laion/ablation-pymethods2test-seqnorm ++trainer.hf_hub_private=false ++trainer.hf_hub_revision=main ++trainer.enable_db_registration=false trainer.project_name=OpenThoughts-Agent trainer.log_level=INFO trainer.tracker_commit_each_step=true trainer.logger=console trainer.run_name=ablation-pymethods2test-seqnorm trainer.ckpt_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/checkpoints trainer.export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/exports trainer.policy.optimizer_config.lr=8e-6 trainer.policy.optimizer_config.weight_decay=0.0 trainer.policy.optimizer_config.adam_betas=[0.9,0.999] trainer.policy.optimizer_config.max_grad_norm=0.9 trainer.policy.fsdp_config.cpu_offload=false trainer.policy.fsdp_config.reshard_after_forward=true trainer.policy.fsdp_config.fsdp_size=4 trainer.policy.model.path=/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 trainer.ref.fsdp_config.cpu_offload=false trainer.ref.fsdp_config.reshard_after_forward=true trainer.ref.fsdp_config.fsdp_size=4 trainer.placement.colocate_all=false trainer.placement.policy_num_nodes=2 trainer.placement.ref_num_nodes=2 trainer.placement.policy_num_gpus_per_node=4 trainer.placement.ref_num_gpus_per_node=4 trainer.fully_async.max_staleness_steps=16 trainer.fully_async.num_parallel_generation_workers=338 generator.backend=vllm generator.timeout_multiplier=1.0 generator.model_dtype=bfloat16 generator.inference_engine_tensor_parallel_size=1 generator.num_inference_engines=48 generator.n_samples_per_prompt=8 generator.eval_n_samples_per_prompt=8 generator.gpu_memory_utilization=0.75 generator.max_num_seqs=24 generator.max_num_batched_tokens=65536 generator.enable_prefix_caching=true generator.enable_chunked_prefill=true generator.run_engines_locally=true generator.weight_sync_backend=nccl generator.async_engine=true generator.batched=false generator.enable_http_endpoint=true generator.enable_ray_prometheus_stats=false generator.vllm_stats_interval=1 generator.append_eos_token_after_stop_str_in_multi_turn=true generator.max_turns=999999 generator.sampling_params.max_generate_length=4096 generator.sampling_params.temperature=0.7 generator.sampling_params.top_p=0.95 generator.sampling_params.top_k=20 ++generator.engine_init_kwargs.max_model_len=32768 ++generator.engine_init_kwargs.custom_chat_template_chat_completion_path=chat_templates/qwen3_thinking_acc.jinja2 ++generator.engine_init_kwargs.served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 data.train_data=["/e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large"] data.val_data=[] +terminal_bench_config.trials_dir=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/trace_jobs +terminal_bench_config.harbor.name=terminus-2 +terminal_bench_config.harbor.max_episodes=999999 +terminal_bench_config.harbor.enable_summarize=false +terminal_bench_config.harbor.store_all_messages=true +terminal_bench_config.harbor.trajectory_config.raw_content=true +terminal_bench_config.harbor.enable_episode_logging=false +terminal_bench_config.harbor.record_terminal_session=false +terminal_bench_config.harbor.enable_pane_logging=false +terminal_bench_config.harbor.strict_json_parser=true +terminal_bench_config.harbor.interleaved_thinking=true +terminal_bench_config.harbor.extra_body.chat_template_kwargs.enable_thinking=true +terminal_bench_config.harbor.override_timeout_sec=900 +terminal_bench_config.harbor.override_cpus=1 +terminal_bench_config.harbor.override_memory_mb=2048 +terminal_bench_config.harbor.override_storage_mb=2048 +terminal_bench_config.harbor.auto_snapshot=true +terminal_bench_config.harbor.verifier_override_timeout_sec=120 +terminal_bench_config.harbor.max_retries=3 +terminal_bench_config.harbor.min_wait_sec=60.0 +terminal_bench_config.harbor.max_wait_sec=600.0 +terminal_bench_config.harbor.wait_multiplier=2.0 +terminal_bench_config.harbor.exclude_exceptions=["VerifierTimeoutError","VerifierRuntimeError","RewardFileNotFoundError","RewardFileEmptyError","VerifierOutputParseError"] +terminal_bench_config.harbor.n_concurrent_trials=675 +terminal_bench_config.harbor.log_level=INFO +terminal_bench_config.harbor.enable_reward_shaping=false +terminal_bench_config.harbor.enable_error_classification=true +terminal_bench_config.harbor.mask_exceptions=["DaytonaError","EnvironmentStartTimeoutError","NetworkError","ConnectionError","RewardFileNotFoundError","RewardFileEmptyError","AgentEnvironmentTimeoutError","ContextLengthExceededError"] +terminal_bench_config.harbor.default_error_treatment=zero +terminal_bench_config.harbor.passthrough_exceptions=["AgentTimeoutError"] +terminal_bench_config.harbor.zero_exceptions=[] +terminal_bench_config.model_info.max_input_tokens=32000 +terminal_bench_config.model_info.max_output_tokens=4096 +terminal_bench_config.archiving.enabled=false +terminal_bench_config.trace_upload.enabled=true +terminal_bench_config.trace_upload.repo_org=DCAgent +terminal_bench_config.trace_upload.episodes=last +terminal_bench_config.trace_upload.dataset_type=SFT +terminal_bench_config.trace_upload.cleanup=true [proxychains] config file found: /e/home/jusers/feuer1/jupiter/.proxychains/proxychains_632525.conf [proxychains] preloading /e/scratch/jureap59/feuer1/proxychains-ng-aarch64/lib/libproxychains4.so [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e 2026-06-06 17:59:59.355 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:623 - Exporting wandb api key to ray runtime env 2026-06-06 17:59:59.355 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:642 - Exporting RAY_ADDRESS to ray runtime env 2026-06-06 17:59:59.355 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:667 - Exporting `NCCL_SOCKET_IFNAME` to ray runtime env: ib0 2026-06-06 17:59:59.355 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:667 - Exporting `NCCL_SOCKET_FAMILY` to ray runtime env: AF_INET 2026-06-06 17:59:59.355 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:667 - Exporting `NCCL_DEBUG` to ray runtime env: WARN 2026-06-06 17:59:59,356 INFO worker.py:1680 -- Using address 10.128.16.35:6379 set in the environment variable RAY_ADDRESS 2026-06-06 17:59:59,385 INFO worker.py:1821 -- Connecting to existing Ray cluster at address: 10.128.16.35:6379... 2026-06-06 17:59:59,394 INFO worker.py:2007 -- Connected to Ray cluster. /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (raylet, ip=10.128.16.187) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e (pid=1435554) [2026-06-06 18:00:00,296 E 1435554 1436307] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 2026-06-06 18:00:01.221 | INFO | skyrl_train.utils.ppo_utils:sync_registries:546 - Synced registries to ray actor (raylet) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 17x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (pid=1435571) [2026-06-06 18:00:06,775 E 1435571 1437247] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 17x across cluster] (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:08.093 | INFO  | skyrl_train.entrypoints.main_base:_configure_log_level:211 - SkyRL log level set to: INFO (skyrl_entrypoint pid=1452083) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:08.358 | INFO  | examples.terminal_bench.dataset:_load_data_files:40 - Loading data from: /e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:10.036 | INFO  | examples.terminal_bench.dataset:_load_data_files:50 - Found 5000 valid task directories out of 5000 total directories (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:10.036 | INFO  | examples.terminal_bench.dataset:__init__:27 - TerminalBenchTaskDataset initialized with 5000 task paths (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:10.048 | INFO  | skyrl_train.entrypoints.main_base:_setup_trainer:394 - data: (skyrl_entrypoint pid=1452083) train_data: (skyrl_entrypoint pid=1452083) - /e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large (skyrl_entrypoint pid=1452083) val_data: [] (skyrl_entrypoint pid=1452083) trainer: (skyrl_entrypoint pid=1452083) placement: (skyrl_entrypoint pid=1452083) colocate_all: false (skyrl_entrypoint pid=1452083) colocate_policy_ref: true (skyrl_entrypoint pid=1452083) policy_num_nodes: 2 (skyrl_entrypoint pid=1452083) policy_num_gpus_per_node: 4 (skyrl_entrypoint pid=1452083) critic_num_nodes: 1 (skyrl_entrypoint pid=1452083) critic_num_gpus_per_node: 4 (skyrl_entrypoint pid=1452083) ref_num_nodes: 2 (skyrl_entrypoint pid=1452083) ref_num_gpus_per_node: 4 (skyrl_entrypoint pid=1452083) policy_strict_spread_pg: false (skyrl_entrypoint pid=1452083) sequence_parallel_backend: ulysses (skyrl_entrypoint pid=1452083) strategy: fsdp2 (skyrl_entrypoint pid=1452083) policy: (skyrl_entrypoint pid=1452083) model: (skyrl_entrypoint pid=1452083) path: /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 (skyrl_entrypoint pid=1452083) lora: (skyrl_entrypoint pid=1452083) rank: 0 (skyrl_entrypoint pid=1452083) alpha: 16 (skyrl_entrypoint pid=1452083) dropout: 0 (skyrl_entrypoint pid=1452083) lora_sync_path: /tmp/skyrl_lora_sync (skyrl_entrypoint pid=1452083) target_modules: all-linear (skyrl_entrypoint pid=1452083) exclude_modules: null (skyrl_entrypoint pid=1452083) deepspeed_config: ${deepspeed_config.train} (skyrl_entrypoint pid=1452083) optimizer_config: (skyrl_entrypoint pid=1452083) optimizer: AdamW (skyrl_entrypoint pid=1452083) lr: 8.0e-06 (skyrl_entrypoint pid=1452083) adam_betas: (skyrl_entrypoint pid=1452083) - 0.9 (skyrl_entrypoint pid=1452083) - 0.999 (skyrl_entrypoint pid=1452083) weight_decay: 0.0 (skyrl_entrypoint pid=1452083) max_grad_norm: 0.9 (skyrl_entrypoint pid=1452083) offload_after_step: true (skyrl_entrypoint pid=1452083) num_warmup_steps: 0 (skyrl_entrypoint pid=1452083) scheduler: constant_with_warmup (skyrl_entrypoint pid=1452083) optimizer_kwargs: {} (skyrl_entrypoint pid=1452083) fsdp_config: (skyrl_entrypoint pid=1452083) cpu_offload: false (skyrl_entrypoint pid=1452083) reshard_after_forward: true (skyrl_entrypoint pid=1452083) fsdp_size: 4 (skyrl_entrypoint pid=1452083) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) expert_tensor_parallel_size: 1 (skyrl_entrypoint pid=1452083) moe_token_dispatcher_type: alltoall (skyrl_entrypoint pid=1452083) moe_router_replay: false (skyrl_entrypoint pid=1452083) moe_grouped_gemm: false (skyrl_entrypoint pid=1452083) ep_comm_backend: torch (skyrl_entrypoint pid=1452083) deepep_num_sms: 20 (skyrl_entrypoint pid=1452083) deepep_token_chunk_size: null (skyrl_entrypoint pid=1452083) sequence_parallel_size: 1 (skyrl_entrypoint pid=1452083) use_torch_compile: false (skyrl_entrypoint pid=1452083) record_memory: false (skyrl_entrypoint pid=1452083) megatron_config: (skyrl_entrypoint pid=1452083) tensor_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) pipeline_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) context_parallel_size: 1 (skyrl_entrypoint pid=1452083) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) expert_tensor_parallel_size: null (skyrl_entrypoint pid=1452083) ddp_config: (skyrl_entrypoint pid=1452083) grad_reduce_in_fp32: true (skyrl_entrypoint pid=1452083) overlap_grad_reduce: false (skyrl_entrypoint pid=1452083) overlap_param_gather: false (skyrl_entrypoint pid=1452083) average_in_collective: true (skyrl_entrypoint pid=1452083) model_config_kwargs: {} (skyrl_entrypoint pid=1452083) torch_profiler_config: (skyrl_entrypoint pid=1452083) enable: false (skyrl_entrypoint pid=1452083) ranks: [] (skyrl_entrypoint pid=1452083) save_path: null (skyrl_entrypoint pid=1452083) optimizer_config_kwargs: (skyrl_entrypoint pid=1452083) overlap_cpu_optimizer_d2h_h2d: false (skyrl_entrypoint pid=1452083) use_precision_aware_optimizer: false (skyrl_entrypoint pid=1452083) optimizer_cpu_offload: false (skyrl_entrypoint pid=1452083) optimizer_offload_fraction: 0.0 (skyrl_entrypoint pid=1452083) transformer_config_kwargs: (skyrl_entrypoint pid=1452083) recompute_granularity: full (skyrl_entrypoint pid=1452083) recompute_modules: (skyrl_entrypoint pid=1452083) - core_attn (skyrl_entrypoint pid=1452083) recompute_method: uniform (skyrl_entrypoint pid=1452083) recompute_num_layers: 1 (skyrl_entrypoint pid=1452083) empty_cuda_cache: true (skyrl_entrypoint pid=1452083) ref: (skyrl_entrypoint pid=1452083) model: (skyrl_entrypoint pid=1452083) path: ${trainer.policy.model.path} (skyrl_entrypoint pid=1452083) sequence_parallel_size: 1 (skyrl_entrypoint pid=1452083) deepspeed_config: ${deepspeed_config.eval} (skyrl_entrypoint pid=1452083) fsdp_config: (skyrl_entrypoint pid=1452083) cpu_offload: false (skyrl_entrypoint pid=1452083) reshard_after_forward: true (skyrl_entrypoint pid=1452083) fsdp_size: 4 (skyrl_entrypoint pid=1452083) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) expert_tensor_parallel_size: 1 (skyrl_entrypoint pid=1452083) moe_token_dispatcher_type: alltoall (skyrl_entrypoint pid=1452083) moe_router_replay: false (skyrl_entrypoint pid=1452083) moe_grouped_gemm: false (skyrl_entrypoint pid=1452083) ep_comm_backend: torch (skyrl_entrypoint pid=1452083) deepep_num_sms: 20 (skyrl_entrypoint pid=1452083) deepep_token_chunk_size: null (skyrl_entrypoint pid=1452083) megatron_config: (skyrl_entrypoint pid=1452083) tensor_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) pipeline_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) context_parallel_size: 1 (skyrl_entrypoint pid=1452083) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) expert_tensor_parallel_size: 1 (skyrl_entrypoint pid=1452083) model_config_kwargs: {} (skyrl_entrypoint pid=1452083) transformer_config_kwargs: {} (skyrl_entrypoint pid=1452083) critic: (skyrl_entrypoint pid=1452083) model: (skyrl_entrypoint pid=1452083) path: null (skyrl_entrypoint pid=1452083) lora: (skyrl_entrypoint pid=1452083) rank: 0 (skyrl_entrypoint pid=1452083) alpha: 16 (skyrl_entrypoint pid=1452083) dropout: 0 (skyrl_entrypoint pid=1452083) target_modules: all-linear (skyrl_entrypoint pid=1452083) exclude_modules: null (skyrl_entrypoint pid=1452083) deepspeed_config: ${deepspeed_config.train} (skyrl_entrypoint pid=1452083) optimizer_config: (skyrl_entrypoint pid=1452083) optimizer: AdamW (skyrl_entrypoint pid=1452083) lr: 5.0e-06 (skyrl_entrypoint pid=1452083) adam_betas: (skyrl_entrypoint pid=1452083) - 0.9 (skyrl_entrypoint pid=1452083) - 0.999 (skyrl_entrypoint pid=1452083) weight_decay: 0.01 (skyrl_entrypoint pid=1452083) max_grad_norm: 1.0 (skyrl_entrypoint pid=1452083) offload_after_step: true (skyrl_entrypoint pid=1452083) num_warmup_steps: 0 (skyrl_entrypoint pid=1452083) scheduler: constant_with_warmup (skyrl_entrypoint pid=1452083) optimizer_kwargs: {} (skyrl_entrypoint pid=1452083) fsdp_config: (skyrl_entrypoint pid=1452083) cpu_offload: false (skyrl_entrypoint pid=1452083) reshard_after_forward: true (skyrl_entrypoint pid=1452083) fsdp_size: -1 (skyrl_entrypoint pid=1452083) expert_model_parallel_size: 1 (skyrl_entrypoint pid=1452083) expert_tensor_parallel_size: 1 (skyrl_entrypoint pid=1452083) moe_token_dispatcher_type: alltoall (skyrl_entrypoint pid=1452083) moe_router_replay: false (skyrl_entrypoint pid=1452083) moe_grouped_gemm: false (skyrl_entrypoint pid=1452083) ep_comm_backend: torch (skyrl_entrypoint pid=1452083) deepep_num_sms: 20 (skyrl_entrypoint pid=1452083) deepep_token_chunk_size: null (skyrl_entrypoint pid=1452083) sequence_parallel_size: 1 (skyrl_entrypoint pid=1452083) algorithm: (skyrl_entrypoint pid=1452083) advantage_estimator: rloo_n (skyrl_entrypoint pid=1452083) kl_ctrl: (skyrl_entrypoint pid=1452083) type: fixed (skyrl_entrypoint pid=1452083) kl_target: 0.1 (skyrl_entrypoint pid=1452083) horizon: 10000 (skyrl_entrypoint pid=1452083) kl_estimator_type: k3 (skyrl_entrypoint pid=1452083) use_kl_estimator_k3: false (skyrl_entrypoint pid=1452083) use_abs_kl: false (skyrl_entrypoint pid=1452083) use_kl_in_reward: false (skyrl_entrypoint pid=1452083) use_kl_loss: false (skyrl_entrypoint pid=1452083) kl_loss_coef: 0.0 (skyrl_entrypoint pid=1452083) use_entropy_loss: false (skyrl_entrypoint pid=1452083) entropy_loss_coef: 0.01 (skyrl_entrypoint pid=1452083) advantage_batch_normalize: false (skyrl_entrypoint pid=1452083) value_head_prefix: value_head (skyrl_entrypoint pid=1452083) policy_loss_type: regular (skyrl_entrypoint pid=1452083) loss_reduction: seq_mean_token_sum_norm_global (skyrl_entrypoint pid=1452083) grpo_norm_by_std: true (skyrl_entrypoint pid=1452083) rloo_n_min_group_size: 4 (skyrl_entrypoint pid=1452083) rloo_n_filter_zero_reward_groups: true (skyrl_entrypoint pid=1452083) lambd: 1.0 (skyrl_entrypoint pid=1452083) gamma: 1.0 (skyrl_entrypoint pid=1452083) eps_clip_low: 0.2 (skyrl_entrypoint pid=1452083) eps_clip_high: 0.05 (skyrl_entrypoint pid=1452083) clip_ratio_c: 3.0 (skyrl_entrypoint pid=1452083) tis_imp_ratio_cap: -1.0 (skyrl_entrypoint pid=1452083) use_tis: false (skyrl_entrypoint pid=1452083) sapo: (skyrl_entrypoint pid=1452083) tau_pos: 1.0 (skyrl_entrypoint pid=1452083) tau_neg: 1.05 (skyrl_entrypoint pid=1452083) value_clip: 0.2 (skyrl_entrypoint pid=1452083) dynamic_sampling: (skyrl_entrypoint pid=1452083) type: null (skyrl_entrypoint pid=1452083) max_sample_batches: 30 (skyrl_entrypoint pid=1452083) min_replace_ratio: 0.3 (skyrl_entrypoint pid=1452083) clip_cov: (skyrl_entrypoint pid=1452083) clip_ratio: 0.0002 (skyrl_entrypoint pid=1452083) clip_cov_lb: 1.0 (skyrl_entrypoint pid=1452083) clip_cov_ub: 5.0 (skyrl_entrypoint pid=1452083) kl_cov: (skyrl_entrypoint pid=1452083) kl_cov_frac: 0.2 (skyrl_entrypoint pid=1452083) ppo_kl_coef: 1.0 (skyrl_entrypoint pid=1452083) cispo: (skyrl_entrypoint pid=1452083) cispo_eps_clip_low: 0 (skyrl_entrypoint pid=1452083) cispo_eps_clip_high: 5 (skyrl_entrypoint pid=1452083) z_clip: (skyrl_entrypoint pid=1452083) enabled: false (skyrl_entrypoint pid=1452083) alpha: 0.97 (skyrl_entrypoint pid=1452083) z_thresh: 2.5 (skyrl_entrypoint pid=1452083) warmup_steps: 25 (skyrl_entrypoint pid=1452083) mode: zscore (skyrl_entrypoint pid=1452083) clip_option: adaptive_scaling (skyrl_entrypoint pid=1452083) clip_factor: 1.0 (skyrl_entrypoint pid=1452083) skip_update_on_spike: false (skyrl_entrypoint pid=1452083) stale_clip: (skyrl_entrypoint pid=1452083) enabled: false (skyrl_entrypoint pid=1452083) alpha: 0.3 (skyrl_entrypoint pid=1452083) entropy_threshold: 0.15 (skyrl_entrypoint pid=1452083) entropy_window: 10 (skyrl_entrypoint pid=1452083) min_lr_scale: 0.1 (skyrl_entrypoint pid=1452083) max_seq_len: 1004095 (skyrl_entrypoint pid=1452083) fully_async: (skyrl_entrypoint pid=1452083) max_staleness_steps: 16 (skyrl_entrypoint pid=1452083) num_parallel_generation_workers: 338 (skyrl_entrypoint pid=1452083) gradient_checkpointing: true (skyrl_entrypoint pid=1452083) gradient_checkpointing_use_reentrant: false (skyrl_entrypoint pid=1452083) seed: 42 (skyrl_entrypoint pid=1452083) resume_mode: latest (skyrl_entrypoint pid=1452083) resume_path: null (skyrl_entrypoint pid=1452083) ckpt_path: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/checkpoints (skyrl_entrypoint pid=1452083) max_ckpts_to_keep: -1 (skyrl_entrypoint pid=1452083) ckpt_interval: 2 (skyrl_entrypoint pid=1452083) hf_save_interval: 5 (skyrl_entrypoint pid=1452083) hf_upload_mode: latest (skyrl_entrypoint pid=1452083) export_path: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/exports (skyrl_entrypoint pid=1452083) bf16: true (skyrl_entrypoint pid=1452083) epochs: 2 (skyrl_entrypoint pid=1452083) max_steps: 80 (skyrl_entrypoint pid=1452083) update_epochs_per_batch: 1 (skyrl_entrypoint pid=1452083) train_batch_size: 64 (skyrl_entrypoint pid=1452083) policy_mini_batch_size: 64 (skyrl_entrypoint pid=1452083) critic_mini_batch_size: 256 (skyrl_entrypoint pid=1452083) micro_train_batch_size_per_gpu: 1 (skyrl_entrypoint pid=1452083) micro_forward_batch_size_per_gpu: 4 (skyrl_entrypoint pid=1452083) update_ref_every_epoch: false (skyrl_entrypoint pid=1452083) use_sample_packing: true (skyrl_entrypoint pid=1452083) eval_batch_size: 64 (skyrl_entrypoint pid=1452083) eval_before_train: false (skyrl_entrypoint pid=1452083) eval_interval: 999999 (skyrl_entrypoint pid=1452083) max_prompt_length: 999999 (skyrl_entrypoint pid=1452083) flash_attn: true (skyrl_entrypoint pid=1452083) disable_fast_tokenizer: false (skyrl_entrypoint pid=1452083) target_modules: null (skyrl_entrypoint pid=1452083) exclude_modules: null (skyrl_entrypoint pid=1452083) project_name: OpenThoughts-Agent (skyrl_entrypoint pid=1452083) run_name: ablation-pymethods2test-seqnorm (skyrl_entrypoint pid=1452083) logger: console (skyrl_entrypoint pid=1452083) tracker_commit_each_step: true (skyrl_entrypoint pid=1452083) dump_data_batch: false (skyrl_entrypoint pid=1452083) dump_eval_results: true (skyrl_entrypoint pid=1452083) log_level: INFO (skyrl_entrypoint pid=1452083) rope_scaling: null (skyrl_entrypoint pid=1452083) rope_theta: null (skyrl_entrypoint pid=1452083) step_wise_training: false (skyrl_entrypoint pid=1452083) hf_hub_repo_id: laion/ablation-pymethods2test-seqnorm (skyrl_entrypoint pid=1452083) hf_hub_private: false (skyrl_entrypoint pid=1452083) hf_hub_revision: main (skyrl_entrypoint pid=1452083) enable_db_registration: false (skyrl_entrypoint pid=1452083) generator: (skyrl_entrypoint pid=1452083) model_name: ${trainer.policy.model.path} (skyrl_entrypoint pid=1452083) model_dtype: bfloat16 (skyrl_entrypoint pid=1452083) timeout_multiplier: 1.0 (skyrl_entrypoint pid=1452083) run_engines_locally: true (skyrl_entrypoint pid=1452083) num_inference_engines: 48 (skyrl_entrypoint pid=1452083) backend: vllm (skyrl_entrypoint pid=1452083) weight_sync_backend: nccl (skyrl_entrypoint pid=1452083) fuse_weights: false (skyrl_entrypoint pid=1452083) weight_transfer_threshold_cuda_ipc_GB: 1.0 (skyrl_entrypoint pid=1452083) inference_engine_tensor_parallel_size: 1 (skyrl_entrypoint pid=1452083) inference_engine_pipeline_parallel_size: 1 (skyrl_entrypoint pid=1452083) inference_engine_expert_parallel_size: 1 (skyrl_entrypoint pid=1452083) inference_engine_data_parallel_size: 1 (skyrl_entrypoint pid=1452083) n_samples_per_prompt: 8 (skyrl_entrypoint pid=1452083) async_engine: true (skyrl_entrypoint pid=1452083) batched: false (skyrl_entrypoint pid=1452083) max_input_length: ${trainer.max_prompt_length} (skyrl_entrypoint pid=1452083) vllm_v1_disable_multiproc: true (skyrl_entrypoint pid=1452083) enable_prefix_caching: true (skyrl_entrypoint pid=1452083) enable_chunked_prefill: true (skyrl_entrypoint pid=1452083) max_num_batched_tokens: 65536 (skyrl_entrypoint pid=1452083) enforce_eager: true (skyrl_entrypoint pid=1452083) fully_sharded_loras: false (skyrl_entrypoint pid=1452083) enable_ray_prometheus_stats: false (skyrl_entrypoint pid=1452083) vllm_stats_interval: 1 (skyrl_entrypoint pid=1452083) gpu_memory_utilization: 0.75 (skyrl_entrypoint pid=1452083) max_num_seqs: 24 (skyrl_entrypoint pid=1452083) remote_inference_engine_urls: (skyrl_entrypoint pid=1452083) - 127.0.0.1:8001 (skyrl_entrypoint pid=1452083) enable_http_endpoint: true (skyrl_entrypoint pid=1452083) http_endpoint_host: 127.0.0.1 (skyrl_entrypoint pid=1452083) http_endpoint_port: 8000 (skyrl_entrypoint pid=1452083) max_turns: 999999 (skyrl_entrypoint pid=1452083) chat_template: (skyrl_entrypoint pid=1452083) source: name (skyrl_entrypoint pid=1452083) name_or_path: null (skyrl_entrypoint pid=1452083) chat_template_kwargs: {} (skyrl_entrypoint pid=1452083) engine_init_kwargs: (skyrl_entrypoint pid=1452083) max_model_len: 32768 (skyrl_entrypoint pid=1452083) custom_chat_template_chat_completion_path: chat_templates/qwen3_thinking_acc.jinja2 (skyrl_entrypoint pid=1452083) served_model_name: 0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6 (skyrl_entrypoint pid=1452083) override_existing_update_group: disable (skyrl_entrypoint pid=1452083) sampling_params: (skyrl_entrypoint pid=1452083) max_generate_length: 4096 (skyrl_entrypoint pid=1452083) repetition_penalty: 1.0 (skyrl_entrypoint pid=1452083) temperature: 0.7 (skyrl_entrypoint pid=1452083) top_p: 0.95 (skyrl_entrypoint pid=1452083) min_p: 0.0 (skyrl_entrypoint pid=1452083) top_k: 20 (skyrl_entrypoint pid=1452083) logprobs: null (skyrl_entrypoint pid=1452083) stop: null (skyrl_entrypoint pid=1452083) use_conversation_multi_turn: true (skyrl_entrypoint pid=1452083) append_eos_token_after_stop_str_in_multi_turn: true (skyrl_entrypoint pid=1452083) eval_sampling_params: (skyrl_entrypoint pid=1452083) max_generate_length: ${generator.sampling_params.max_generate_length} (skyrl_entrypoint pid=1452083) repetition_penalty: 1.0 (skyrl_entrypoint pid=1452083) temperature: 0.0 (skyrl_entrypoint pid=1452083) top_p: 1.0 (skyrl_entrypoint pid=1452083) min_p: 0.0 (skyrl_entrypoint pid=1452083) top_k: -1 (skyrl_entrypoint pid=1452083) logprobs: null (skyrl_entrypoint pid=1452083) stop: null (skyrl_entrypoint pid=1452083) eval_n_samples_per_prompt: 8 (skyrl_entrypoint pid=1452083) zero_reward_on_non_stop: false (skyrl_entrypoint pid=1452083) apply_overlong_filtering: false (skyrl_entrypoint pid=1452083) rope_scaling: ${trainer.rope_scaling} (skyrl_entrypoint pid=1452083) rope_theta: ${trainer.rope_theta} (skyrl_entrypoint pid=1452083) teacher: (skyrl_entrypoint pid=1452083) model_path: null (skyrl_entrypoint pid=1452083) top_k_logprobs: 256 (skyrl_entrypoint pid=1452083) num_inference_engines: 1 (skyrl_entrypoint pid=1452083) inference_engine_tensor_parallel_size: 1 (skyrl_entrypoint pid=1452083) inference_engine_pipeline_parallel_size: 1 (skyrl_entrypoint pid=1452083) gpu_memory_utilization: 0.9 (skyrl_entrypoint pid=1452083) enforce_eager: false (skyrl_entrypoint pid=1452083) backend: vllm (skyrl_entrypoint pid=1452083) engine_init_kwargs: {} (skyrl_entrypoint pid=1452083) environment: (skyrl_entrypoint pid=1452083) env_class: gsm8k (skyrl_entrypoint pid=1452083) skyrl_gym: (skyrl_entrypoint pid=1452083) max_env_workers: 32 (skyrl_entrypoint pid=1452083) text2sql: (skyrl_entrypoint pid=1452083) db_path: /home/ray/default/sql_data (skyrl_entrypoint pid=1452083) llm_as_a_judge: (skyrl_entrypoint pid=1452083) model: gpt-4o-mini (skyrl_entrypoint pid=1452083) base_url: null (skyrl_entrypoint pid=1452083) search: (skyrl_entrypoint pid=1452083) log_requests: false (skyrl_entrypoint pid=1452083) search_url: http://127.0.0.1:8000/retrieve (skyrl_entrypoint pid=1452083) topk: 3 (skyrl_entrypoint pid=1452083) timeout: 30 (skyrl_entrypoint pid=1452083) rollout: (skyrl_entrypoint pid=1452083) fanout: (skyrl_entrypoint pid=1452083) enabled: true (skyrl_entrypoint pid=1452083) num_coordinators: 4 (skyrl_entrypoint pid=1452083) cpus_per_coordinator: 8 (skyrl_entrypoint pid=1452083) deepspeed_config: (skyrl_entrypoint pid=1452083) train: (skyrl_entrypoint pid=1452083) zero_optimization: (skyrl_entrypoint pid=1452083) stage: 3 (skyrl_entrypoint pid=1452083) offload_param: (skyrl_entrypoint pid=1452083) device: none (skyrl_entrypoint pid=1452083) offload_optimizer: (skyrl_entrypoint pid=1452083) device: none (skyrl_entrypoint pid=1452083) pin_memory: true (skyrl_entrypoint pid=1452083) sub_group_size: auto (skyrl_entrypoint pid=1452083) reduce_bucket_size: auto (skyrl_entrypoint pid=1452083) stage3_param_persistence_threshold: auto (skyrl_entrypoint pid=1452083) stage3_prefetch_bucket_size: auto (skyrl_entrypoint pid=1452083) stage3_max_live_parameters: auto (skyrl_entrypoint pid=1452083) stage3_max_reuse_distance: auto (skyrl_entrypoint pid=1452083) round_robin_gradients: true (skyrl_entrypoint pid=1452083) zero_hpz_partition_size: 1 (skyrl_entrypoint pid=1452083) zero_quantized_weights: false (skyrl_entrypoint pid=1452083) zero_quantized_gradients: false (skyrl_entrypoint pid=1452083) torch_autocast: (skyrl_entrypoint pid=1452083) enabled: true (skyrl_entrypoint pid=1452083) dtype: bfloat16 (skyrl_entrypoint pid=1452083) disable_trace_cache: false (skyrl_entrypoint pid=1452083) data_types: (skyrl_entrypoint pid=1452083) grad_accum_dtype: fp32 (skyrl_entrypoint pid=1452083) gradient_clipping: 1.0 (skyrl_entrypoint pid=1452083) wall_clock_breakdown: false (skyrl_entrypoint pid=1452083) prescale_gradient: false (skyrl_entrypoint pid=1452083) eval: (skyrl_entrypoint pid=1452083) zero_optimization: (skyrl_entrypoint pid=1452083) stage: 3 (skyrl_entrypoint pid=1452083) stage3_param_persistence_threshold: auto (skyrl_entrypoint pid=1452083) offload_param: (skyrl_entrypoint pid=1452083) device: cpu (skyrl_entrypoint pid=1452083) pin_memory: true (skyrl_entrypoint pid=1452083) torch_autocast: (skyrl_entrypoint pid=1452083) enabled: true (skyrl_entrypoint pid=1452083) dtype: bfloat16 (skyrl_entrypoint pid=1452083) gradient_clipping: 1.0 (skyrl_entrypoint pid=1452083) prescale_gradient: false (skyrl_entrypoint pid=1452083) wall_clock_breakdown: false (skyrl_entrypoint pid=1452083) terminal_bench_config: (skyrl_entrypoint pid=1452083) trials_dir: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/trace_jobs (skyrl_entrypoint pid=1452083) harbor: (skyrl_entrypoint pid=1452083) name: terminus-2 (skyrl_entrypoint pid=1452083) max_episodes: 999999 (skyrl_entrypoint pid=1452083) enable_summarize: false (skyrl_entrypoint pid=1452083) store_all_messages: true (skyrl_entrypoint pid=1452083) trajectory_config: (skyrl_entrypoint pid=1452083) raw_content: true (skyrl_entrypoint pid=1452083) enable_episode_logging: false (skyrl_entrypoint pid=1452083) record_terminal_session: false (skyrl_entrypoint pid=1452083) enable_pane_logging: false (skyrl_entrypoint pid=1452083) strict_json_parser: true (skyrl_entrypoint pid=1452083) interleaved_thinking: true (skyrl_entrypoint pid=1452083) extra_body: (skyrl_entrypoint pid=1452083) chat_template_kwargs: (skyrl_entrypoint pid=1452083) enable_thinking: true (skyrl_entrypoint pid=1452083) override_timeout_sec: 900 (skyrl_entrypoint pid=1452083) override_cpus: 1 (skyrl_entrypoint pid=1452083) override_memory_mb: 2048 (skyrl_entrypoint pid=1452083) override_storage_mb: 2048 (skyrl_entrypoint pid=1452083) auto_snapshot: true (skyrl_entrypoint pid=1452083) verifier_override_timeout_sec: 120 (skyrl_entrypoint pid=1452083) max_retries: 3 (skyrl_entrypoint pid=1452083) min_wait_sec: 60.0 (skyrl_entrypoint pid=1452083) max_wait_sec: 600.0 (skyrl_entrypoint pid=1452083) wait_multiplier: 2.0 (skyrl_entrypoint pid=1452083) exclude_exceptions: (skyrl_entrypoint pid=1452083) - VerifierTimeoutError (skyrl_entrypoint pid=1452083) - VerifierRuntimeError (skyrl_entrypoint pid=1452083) - RewardFileNotFoundError (skyrl_entrypoint pid=1452083) - RewardFileEmptyError (skyrl_entrypoint pid=1452083) - VerifierOutputParseError (skyrl_entrypoint pid=1452083) n_concurrent_trials: 675 (skyrl_entrypoint pid=1452083) log_level: INFO (skyrl_entrypoint pid=1452083) enable_reward_shaping: false (skyrl_entrypoint pid=1452083) enable_error_classification: true (skyrl_entrypoint pid=1452083) mask_exceptions: (skyrl_entrypoint pid=1452083) - DaytonaError (skyrl_entrypoint pid=1452083) - EnvironmentStartTimeoutError (skyrl_entrypoint pid=1452083) - NetworkError (skyrl_entrypoint pid=1452083) - ConnectionError (skyrl_entrypoint pid=1452083) - RewardFileNotFoundError (skyrl_entrypoint pid=1452083) - RewardFileEmptyError (skyrl_entrypoint pid=1452083) - AgentEnvironmentTimeoutError (skyrl_entrypoint pid=1452083) - ContextLengthExceededError (skyrl_entrypoint pid=1452083) default_error_treatment: zero (skyrl_entrypoint pid=1452083) passthrough_exceptions: (skyrl_entrypoint pid=1452083) - AgentTimeoutError (skyrl_entrypoint pid=1452083) zero_exceptions: [] (skyrl_entrypoint pid=1452083) model_info: (skyrl_entrypoint pid=1452083) max_input_tokens: 32000 (skyrl_entrypoint pid=1452083) max_output_tokens: 4096 (skyrl_entrypoint pid=1452083) archiving: (skyrl_entrypoint pid=1452083) enabled: false (skyrl_entrypoint pid=1452083) trace_upload: (skyrl_entrypoint pid=1452083) enabled: true (skyrl_entrypoint pid=1452083) repo_org: DCAgent (skyrl_entrypoint pid=1452083) episodes: last (skyrl_entrypoint pid=1452083) dataset_type: SFT (skyrl_entrypoint pid=1452083) cleanup: true (skyrl_entrypoint pid=1452083)  (skyrl_entrypoint pid=1452083) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: (skyrl_entrypoint pid=1452083) No module named 'vllm._version' (skyrl_entrypoint pid=1452083) from .version import __version__, __version_tuple__ # isort:skip (skyrl_entrypoint pid=1452083) W0606 18:00:14.997000 1452083 envs/rl/lib/python3.12/site-packages/torch/utils/cpp_extension.py:117] No CUDA runtime is found, using CUDA_HOME='/e/software/default/stages/2026/software/CUDA/13' (pid=1435835) [2026-06-06 18:00:10,011 E 1435835 1451080] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 268x across cluster] (raylet) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e (raylet, ip=10.128.16.45) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 43x across cluster] (pid=1458206, ip=10.128.16.42) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: (pid=1458206, ip=10.128.16.42) No module named 'vllm._version' (pid=1458206, ip=10.128.16.42) from .version import __version__, __version_tuple__ # isort:skip (raylet, ip=10.128.16.37) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 109x across cluster] (pid=2131461, ip=10.128.16.185) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 7x across cluster] (pid=2131461, ip=10.128.16.185) No module named 'vllm._version' [repeated 7x across cluster] (pid=2131461, ip=10.128.16.185) from .version import __version__, __version_tuple__ # isort:skip [repeated 7x across cluster] [2026-06-06 18:00:29,592 E 1451735 1452057] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (RegistryActor pid=1953416, ip=10.128.16.187) [2026-06-06 18:00:30,303 E 1953416 1953456] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:30.445 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:123 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=0, VLLM_ENABLE_V1_MULTIPROCESSING= (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:30.446 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:30.446 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:31.096 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 0-71 (NUMA node 0) for GPU 0 (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:31.121 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 0 (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:31.121 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:584 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:31.121 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:592 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:31.121 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1158 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) 2026-06-06 18:00:31.151 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1210 - Engine startup stagger: sleeping 1.70s (attempt 1/5) to avoid port collisions (raylet, ip=10.128.16.38) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 81x across cluster] (pid=1455593, ip=10.128.16.46) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 17x across cluster] (pid=1455593, ip=10.128.16.46) No module named 'vllm._version' [repeated 17x across cluster] (pid=1455593, ip=10.128.16.46) from .version import __version__, __version_tuple__ # isort:skip [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (skyrl_entrypoint pid=1452083) [2026-06-06 18:00:32,209 E 1452083 1452128] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:34.675 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:123 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=3, VLLM_ENABLE_V1_MULTIPROCESSING= [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:34.675 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:34.675 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:35.618 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 216-287 (NUMA node 3) for GPU 3 [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:35.622 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 3 [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:35.622 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:584 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:35.623 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:592 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:35.623 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1158 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) 2026-06-06 18:00:35.651 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1210 - Engine startup stagger: sleeping 2.78s (attempt 1/5) to avoid port collisions [repeated 7x across cluster] (raylet, ip=10.128.16.187) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 122x across cluster] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) No module named 'vllm._version' [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) from .version import __version__, __version_tuple__ # isort:skip [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 5x across cluster] (skyrl_entrypoint pid=1452083) [2026-06-06 18:00:38] INFO inference_engine_client_http_endpoint.py:350: Starting server on 0.0.0.0:8000 (skyrl_entrypoint pid=1452083) [2026-06-06 18:00:38] INFO inference_engine_client_http_endpoint.py:242: Starting inference HTTP endpoint... (skyrl_entrypoint pid=1452083) [2026-06-06 18:00:39] INFO inference_engine_client_http_endpoint.py:229: Server ready after 2 attempts (2 seconds) (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.344 | INFO  | skyrl_train.inference_engines.inference_engine_client:_spin_up_http_endpoint:969 - InferenceEngineClient HTTP endpoint started on 127.0.0.1:8000 (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.344 | INFO  | skyrl_train.inference_engines.inference_engine_client:__init__:61 - InferenceEngineClient initialized with 48 engines. (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.346 | INFO  | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:295 - Harbor logging level set to INFO (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.347 | INFO  | examples.terminal_bench.terminal_bench_generator:__init__:195 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 675. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.348 | INFO  | examples.terminal_bench.terminal_bench_generator:__init__:211 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.349 | INFO  | skyrl_train.utils.trainer_utils:build_dataloader:656 - Total steps: 156 (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.349 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:357 - Length of train_dataloader: 5000 (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.349 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:358 - Number of steps per epoch: 78 (skyrl_entrypoint pid=1452083) 2026-06-06 18:00:39.349 | INFO  | skyrl_train.fully_async_trainer:_build_train_dataloader_and_compute_training_steps:359 - Total training steps: 80 (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) 2026-06-06 18:00:40.441 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:123 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=0, VLLM_ENABLE_V1_MULTIPROCESSING= [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) 2026-06-06 18:00:40.443 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) 2026-06-06 18:00:40.443 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) 2026-06-06 18:00:40.006 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 216-287 (NUMA node 3) for GPU 3 [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) 2026-06-06 18:00:40.029 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 3 [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) 2026-06-06 18:00:40.029 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:584 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) 2026-06-06 18:00:40.029 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:592 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) 2026-06-06 18:00:40.029 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1158 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) 2026-06-06 18:00:40.060 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1210 - Engine startup stagger: sleeping 1.92s (attempt 1/5) to avoid port collisions [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 59x across cluster] (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 21x across cluster] (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) No module named 'vllm._version' [repeated 21x across cluster] (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) from .version import __version__, __version_tuple__ # isort:skip [repeated 21x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 30x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 15x across cluster] (pid=1464878, ip=10.128.16.48) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/__init__.py:1617: UserWarning: Please use the new API settings to control TF32 behavior, such as torch.backends.cudnn.conv.fp32_precision = 'tf32' or torch.backends.cuda.matmul.fp32_precision = 'ieee'. Old settings, e.g, torch.backends.cuda.matmul.allow_tf32 = True, torch.backends.cudnn.allow_tf32 = True, allowTF32CuDNN() and allowTF32CuBLAS() will be deprecated after Pytorch 2.9. Please see https://pytorch.org/docs/main/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:80.) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) _C._set_float32_matmul_precision(precision) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) [W606 18:00:44.932301925 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-001-42.jupiter.internal]:45389 (errno: 97 - Address family not supported by protocol). (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) [W606 18:00:44.932909847 Utils.hpp:166] Warning: Environment variable NCCL_BLOCKING_WAIT is deprecated; use TORCH_NCCL_BLOCKING_WAIT instead (function operator()) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) [rank0]:[W606 18:00:44.935505181 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:43.713 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:123 - setup_envvars_for_vllm: distributed_executor_backend=uni, SKYRL_ENABLE_NUMA_AFFINITY=1, CUDA_VISIBLE_DEVICES=1, VLLM_ENABLE_V1_MULTIPROCESSING= [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:43.714 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:43.714 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:44.675 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 72-143 (NUMA node 1) for GPU 1 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:44.680 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 1 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:44.680 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:584 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:44.680 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:592 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:44.680 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1158 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) 2026-06-06 18:00:44.713 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1210 - Engine startup stagger: sleeping 2.86s (attempt 1/5) to avoid port collisions [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) 2026-06-06 18:00:49.427 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:133 - setup_envvars_for_vllm: numa_enabled=True [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) 2026-06-06 18:00:49.427 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:setup_envvars_for_vllm:136 - setup_envvars_for_vllm: set VLLM_ENABLE_V1_MULTIPROCESSING=0 for NUMA affinity [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) (EngineCore_DP0 pid=2131687) _C._set_float32_matmul_precision(precision) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) (EngineCore_DP0 pid=1458588) _C._set_float32_matmul_precision(precision) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) 2026-06-06 18:00:50.679 | INFO | skyrl_train.utils.numa:set_numa_affinity_for_gpu:340 - NUMA affinity: bound process to CPUs 144-215 (NUMA node 2) for GPU 2 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) 2026-06-06 18:00:50.703 | DEBUG | skyrl_train.utils.numa:_set_membind_via_libnuma:375 - NUMA affinity: set memory preferred to NUMA node 2 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) 2026-06-06 18:00:50.703 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:584 - BaseVLLMInferenceEngine: vllm_v1_disable_multiproc=True, vllm.__version__=dev, VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) 2026-06-06 18:00:50.703 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:__init__:592 - BaseVLLMInferenceEngine: set VLLM_ENABLE_V1_MULTIPROCESSING=0 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) 2026-06-06 18:00:50.703 | WARNING | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1158 - OpenAI API sampling params overridden: temperature=0.7, top_p=0.95, top_k=20 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) 2026-06-06 18:00:50.733 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1210 - Engine startup stagger: sleeping 2.08s (attempt 1/5) to avoid port collisions [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) (EngineCore_DP0 pid=1458572) Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00 10.128.16.35 (routable head IP) for coordinator litellm base_url connectivity (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:20.082 | INFO  | examples.terminal_bench.rollout_coordinator:__init__:405 - [RolloutDispatcher] configured num_coordinators=4, cpus_per_coordinator=8 (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:20.102 | INFO  | examples.terminal_bench.rollout_coordinator:startup:434 - [RolloutDispatcher] PlacementGroup ready: 4 bundles x 8 CPU (SPREAD) (raylet, ip=10.128.16.36) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e (FSDPPolicyWorkerBase pid=1452251) [2026-06-06 18:01:21,183 E 1452251 1452439] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) (EngineCore_DP0 pid=1953854) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) (EngineCore_DP0 pid=1953841) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) (EngineCore_DP0 pid=1953862) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) (EngineCore_DP0 pid=1101804) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 19x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) 2026-06-06 18:01:24.633 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) 2026-06-06 18:01:22.323 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1267 - Initializing OpenAIServingChat with custom_chat_template read from: chat_templates/qwen3_thinking_acc.jinja2 [repeated 19x across cluster] (FSDPPolicyWorkerBase pid=1464952, ip=10.128.16.48) [rank1]:[W606 18:01:17.522842374 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:13<00:00, 3.39s/it] [repeated 20x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) 2026-06-06 18:01:24.846 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:295 - Harbor logging level set to INFO (RolloutCoordinator pid=1437455, ip=10.128.16.36) 2026-06-06 18:01:24.846 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:195 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (RolloutCoordinator pid=1437455, ip=10.128.16.36) 2026-06-06 18:01:24.846 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:211 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (RolloutCoordinator pid=1437455, ip=10.128.16.36) 2026-06-06 18:01:24.846 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 0/4] constructed (http=10.128.16.35:8000) (RolloutCoordinator pid=1437455, ip=10.128.16.36) 2026-06-06 18:01:24.849 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:351 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError (RolloutCoordinator pid=1437455, ip=10.128.16.36) 2026-06-06 18:01:24.849 | INFO | examples.terminal_bench.terminal_bench_generator:startup:308 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168 (RolloutCoordinator pid=1437455, ip=10.128.16.36) 2026-06-06 18:01:24.849 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 0] startup complete (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:24.850 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 1/4 started (raylet, ip=10.128.16.36) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 5x across cluster] (FSDPPolicyWorkerBase pid=1464952, ip=10.128.16.48) [2026-06-06 18:01:21,244 E 1464952 1465018] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) 2026-06-06 18:01:31.611 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) 2026-06-06 18:01:25.341 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:_create_engine:1267 - Initializing OpenAIServingChat with custom_chat_template read from: chat_templates/qwen3_thinking_acc.jinja2 [repeated 4x across cluster] (raylet, ip=10.128.16.38) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster] (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:31.832 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 2/4 started (RolloutCoordinator pid=1102057, ip=10.128.16.38) 2026-06-06 18:01:31.828 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:295 - Harbor logging level set to INFO (RolloutCoordinator pid=1102057, ip=10.128.16.38) 2026-06-06 18:01:31.828 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:195 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (RolloutCoordinator pid=1102057, ip=10.128.16.38) 2026-06-06 18:01:31.829 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:211 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (RolloutCoordinator pid=1102057, ip=10.128.16.38) 2026-06-06 18:01:31.829 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 1/4] constructed (http=10.128.16.35:8000) (RolloutCoordinator pid=1102057, ip=10.128.16.38) 2026-06-06 18:01:31.831 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:351 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError (RolloutCoordinator pid=1102057, ip=10.128.16.38) 2026-06-06 18:01:31.831 | INFO | examples.terminal_bench.terminal_bench_generator:startup:308 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168 (RolloutCoordinator pid=1102057, ip=10.128.16.38) 2026-06-06 18:01:31.831 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 1] startup complete (RolloutCoordinator pid=1102057, ip=10.128.16.38) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (RolloutCoordinator pid=1429749, ip=10.128.16.41) 2026-06-06 18:01:40.747 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4) (raylet, ip=10.128.16.41) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster] (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:40.974 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 3/4 started (RolloutCoordinator pid=1429749, ip=10.128.16.41) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (RolloutCoordinator pid=1429749, ip=10.128.16.41) 2026-06-06 18:01:40.970 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:295 - Harbor logging level set to INFO (RolloutCoordinator pid=1429749, ip=10.128.16.41) 2026-06-06 18:01:40.971 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:195 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (RolloutCoordinator pid=1429749, ip=10.128.16.41) 2026-06-06 18:01:40.971 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:211 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (RolloutCoordinator pid=1429749, ip=10.128.16.41) 2026-06-06 18:01:40.971 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 2/4] constructed (http=10.128.16.35:8000) (RolloutCoordinator pid=1429749, ip=10.128.16.41) 2026-06-06 18:01:40.973 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:351 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError (RolloutCoordinator pid=1429749, ip=10.128.16.41) 2026-06-06 18:01:40.974 | INFO | examples.terminal_bench.terminal_bench_generator:startup:308 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168 (RolloutCoordinator pid=1429749, ip=10.128.16.41) 2026-06-06 18:01:40.974 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 2] startup complete (RolloutCoordinator pid=1388832, ip=10.128.16.44) 2026-06-06 18:01:48.659 | INFO | examples.terminal_bench.rollout_coordinator:_scale_terminal_bench_cfg:121 - [RolloutCoordinator] scaled n_concurrent_trials 675 -> 168 (// 4) (raylet, ip=10.128.16.44) [proxychains] DLL init: proxychains-ng 4.17-git-9-g78ead0e [repeated 6x across cluster] (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:48.895 | INFO  | examples.terminal_bench.rollout_coordinator:startup:469 - [RolloutDispatcher] coordinator 4/4 started (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:48.895 | INFO  | examples.terminal_bench.rollout_coordinator:startup:477 - [RolloutDispatcher] 4 coordinators started (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:48.895 | INFO  | skyrl_train.fully_async_trainer:train:485 - Generator startup complete (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:48.895 | INFO  | skyrl_train.fully_async_trainer:_train_loop:510 - Started: 'load_checkpoints' (RolloutCoordinator pid=1388832, ip=10.128.16.44) The tokenizer you are loading from '/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue. (RolloutCoordinator pid=1388832, ip=10.128.16.44) 2026-06-06 18:01:48.890 | INFO | examples.terminal_bench.terminal_bench_generator:_configure_harbor_logging:295 - Harbor logging level set to INFO (RolloutCoordinator pid=1388832, ip=10.128.16.44) 2026-06-06 18:01:48.890 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:195 - TerminalBenchGenerator initialized with HarborConfigBuilder. Exposed fields: ['name', 'max_episodes', 'enable_summarize', 'store_all_messages', 'trajectory_config', 'enable_episode_logging', 'record_terminal_session', 'enable_pane_logging', 'strict_json_parser', 'interleaved_thinking', 'extra_body', 'override_timeout_sec', 'override_cpus', 'override_memory_mb', 'override_storage_mb', 'auto_snapshot', 'verifier_override_timeout_sec', 'max_retries', 'min_wait_sec', 'max_wait_sec', 'wait_multiplier', 'exclude_exceptions', 'n_concurrent_trials', 'log_level', 'enable_reward_shaping', 'enable_error_classification', 'mask_exceptions', 'default_error_treatment', 'passthrough_exceptions', 'zero_exceptions']. Retry config: max_retries=3, backoff=60.0-600.0s. Concurrent trials: 168. Reward shaping: enabled=False, shaper=pass_ratio. Error classification: enabled=True (RolloutCoordinator pid=1388832, ip=10.128.16.44) 2026-06-06 18:01:48.891 | INFO | examples.terminal_bench.terminal_bench_generator:__init__:211 - TerminalBenchGenerator initialized with custom chat template read from: chat_templates/qwen3_thinking_acc.jinja2 (RolloutCoordinator pid=1388832, ip=10.128.16.44) 2026-06-06 18:01:48.891 | INFO | examples.terminal_bench.rollout_coordinator:__init__:246 - [RolloutCoordinator 3/4] constructed (http=10.128.16.35:8000) (RolloutCoordinator pid=1388832, ip=10.128.16.44) 2026-06-06 18:01:48.893 | INFO | examples.terminal_bench.terminal_bench_generator:_create_orchestrator:351 - QueueOrchestrator created and started with n_concurrent_trials=168, rollback_hook registered for ContextLengthExceededError/AgentTimeoutError (RolloutCoordinator pid=1388832, ip=10.128.16.44) 2026-06-06 18:01:48.894 | INFO | examples.terminal_bench.terminal_bench_generator:startup:308 - TerminalBenchGenerator startup complete. Shared orchestrator ready with n_concurrent_trials=168 (RolloutCoordinator pid=1388832, ip=10.128.16.44) 2026-06-06 18:01:48.894 | INFO | examples.terminal_bench.rollout_coordinator:startup:254 - [RolloutCoordinator 3] startup complete (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:48.896 | INFO  | skyrl_train.trainer:load_checkpoints:1628 - No checkpoint found, starting training from scratch (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:48.896 | INFO  | skyrl_train.fully_async_trainer:_train_loop:512 - Resumed training from global_step 0 (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:48.896 | INFO  | skyrl_train.fully_async_trainer:_train_loop:510 - Finished: 'load_checkpoints', time cost: 0.00s (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:48.896 | INFO  | skyrl_train.fully_async_trainer:_train_loop:544 - Started: 'init_weight_sync_state' (FSDPPolicyWorkerBase pid=1464878, ip=10.128.16.48) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: (FSDPPolicyWorkerBase pid=1464878, ip=10.128.16.48) No module named 'vllm._version' (FSDPPolicyWorkerBase pid=1464878, ip=10.128.16.48) from .version import __version__, __version_tuple__ # isort:skip (RolloutCoordinator pid=1437455, ip=10.128.16.36) [2026-06-06 18:01:50,843 E 1437455 1437495] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (FSDPPolicyWorkerBase pid=1464878, ip=10.128.16.48) 2026-06-06 18:01:54.064 | INFO  | logging:info:2216 - [weight-sync] Using master_addr=10.128.16.48, master_port=45245 (FSDPPolicyWorkerBase pid=1464878, ip=10.128.16.48) 2026-06-06 18:01:54.063 | INFO  | logging:info:2216 - [weight-sync] get_node_ip_address()=10.128.16.48 [repeated 3x across cluster] (FSDPPolicyWorkerBase pid=1452250) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/__init__.py:7: RuntimeWarning: Failed to read commit hash: [repeated 7x across cluster] (FSDPPolicyWorkerBase pid=1452250) No module named 'vllm._version' [repeated 7x across cluster] (FSDPPolicyWorkerBase pid=1452250) from .version import __version__, __version_tuple__ # isort:skip [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) (EngineCore_DP0 pid=1455823) 2026-06-06 18:01:54.221 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:init_weight_update_communicator:189 - torch.distributed.get_rank(): 0, rank_offset: 21, rank: 21, world_size: 49, group_name: skyrl (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) [rank0]:[W606 18:01:54.615757335 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [jpbo-001-48-interconnect-1.jupiter.internal]:45245 (errno: 97 - Address family not supported by protocol). (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) (EngineCore_DP0 pid=1387812) 2026-06-06 18:01:54.223 | INFO | skyrl_train.inference_engines.vllm.vllm_engine:init_weight_update_communicator:200 - init_weight_update_communicator: master_address=10.128.16.48, master_port=45245, (FSDPPolicyWorkerBase pid=1464878, ip=10.128.16.48) /e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/torch/distributed/distributed_c10d.py:4876: UserWarning: barrier(): using the device under current context. You can specify `device_id` in `init_process_group` to mute this warning. (FSDPPolicyWorkerBase pid=1464878, ip=10.128.16.48) warnings.warn( # warn only once (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:54.323 | INFO  | skyrl_train.trainer:init_weight_sync_state:847 - Initialized weight sync state for policy model and inference engines. (skyrl_entrypoint pid=1452083) 2026-06-06 18:01:54.323 | INFO  | skyrl_train.fully_async_trainer:_train_loop:544 - Finished: 'init_weight_sync_state', time cost: 5.43s (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) [rank0]:[W606 18:01:57.709020420 Utils.hpp:137] Warning: Environment variable TORCH_NCCL_TRACE_BUFFER_SIZE is deprecated; use TORCH_FR_BUFFER_SIZE instead (function operator()) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [2026-06-06 18:01:57,635 E 1102057 1102097] core_worker_process.cc:842: Failed to establish connection to the metrics exporter agent. Metrics will not be exported. Exporter agent status: RpcError: Running out of retries to initialize the metrics agent. rpc_code: 14 (skyrl_entrypoint pid=1452083) 2026-06-06 18:02:04.621 | INFO  | skyrl_train.fully_async_trainer:_train_loop:548 - Finished: 'sync_weights_to_inference_engines', time cost: 10.30s (skyrl_entrypoint pid=1452083) 2026-06-06 18:02:04.623 | INFO  | skyrl_train.callbacks.builtin:on_train_begin:270 - HFHubUploadCallback initialized: repo=laion/ablation-pymethods2test-seqnorm, upload_steps=5, export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/exports (skyrl_entrypoint pid=1452083) Training Step Progress: 0%| | 0/80 [00:00, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} (pid=1388353, ip=10.128.16.44) INFO 06-06 18:00:40 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) WARNING 06-06 18:00:42 [arg_utils.py:1256] The global random seed is set to 50. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:00:42 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:00:42 [model.py:1549] Using max model len 32768 [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:00:42 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:00:42 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:00:42 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) WARNING 06-06 18:00:42 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:00:42 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) WARNING 06-06 18:00:42 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) WARNING 06-06 18:00:42 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:00:44 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:00:44 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.42:45389 backend=nccl (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:00:45 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) (EngineCore_DP0 pid=1246806) INFO 06-06 18:00:45 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=44, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:00:45 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:00:46 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) INFO 06-06 18:00:46 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) WARNING 06-06 18:00:47 [arg_utils.py:1256] The global random seed is set to 79. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) INFO 06-06 18:00:47 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) INFO 06-06 18:00:47 [model.py:1549] Using max model len 32768 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) INFO 06-06 18:00:47 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) INFO 06-06 18:00:47 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) INFO 06-06 18:00:47 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) WARNING 06-06 18:00:47 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) INFO 06-06 18:00:47 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) WARNING 06-06 18:00:47 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) WARNING 06-06 18:00:47 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) (EngineCore_DP0 pid=1246816) INFO 06-06 18:00:49 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) (EngineCore_DP0 pid=1246816) INFO 06-06 18:00:49 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.45:59735 backend=nccl [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) (EngineCore_DP0 pid=1246816) INFO 06-06 18:00:49 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 4x across cluster] (pid=1464878, ip=10.128.16.48) ⚙️ Running in WANDB offline mode (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) (EngineCore_DP0 pid=1387816) INFO 06-06 18:00:50 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=61, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) (EngineCore_DP0 pid=1246816) INFO 06-06 18:00:50 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) (EngineCore_DP0 pid=1458588) INFO 06-06 18:00:51 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:00:50 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) WARNING 06-06 18:00:53 [arg_utils.py:1256] The global random seed is set to 89. Since VLLM_ENABLE_V1_MULTIPROCESSING is set to False, this may affect the random state of the Python process that launched vLLM. [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:00:53 [model.py:529] Resolved architecture: Qwen3ForCausalLM [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:00:53 [model.py:1549] Using max model len 32768 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:00:53 [arg_utils.py:1469] Using ray runtime env (env vars redacted): {'env_vars': {'NCCL_CUMEM_ENABLE': '***', 'NCCL_DEBUG': '***', 'NCCL_SOCKET_FAMILY': '***', 'NCCL_SOCKET_IFNAME': '***', 'RAY_ADDRESS': '***', 'VLLM_ALLOW_INSECURE_SERIALIZATION': '***', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': '***', 'VLLM_DISABLE_COMPILE_CACHE': '***', 'WANDB_API_KEY': '***'}} [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:00:53 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=65536. [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:00:53 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) WARNING 06-06 18:00:53 [vllm.py:728] Enforce eager set, overriding optimization level to -O0 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:00:53 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) WARNING 06-06 18:00:53 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) WARNING 06-06 18:00:53 [system_utils.py:140] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) (EngineCore_DP0 pid=1387832) INFO 06-06 18:00:53 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) (EngineCore_DP0 pid=1387816) INFO 06-06 18:00:54 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.37:48183 backend=nccl [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) (EngineCore_DP0 pid=1387828) INFO 06-06 18:00:55 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) (EngineCore_DP0 pid=1429601) INFO 06-06 18:00:55 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=80, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) (EngineCore_DP0 pid=1387828) INFO 06-06 18:00:55 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) (EngineCore_DP0 pid=1387816) INFO 06-06 18:00:56 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 9x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:00:56 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 16x across cluster] (pid=1464951, ip=10.128.16.48) ⚙️ Running in WANDB offline mode (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) (EngineCore_DP0 pid=1437257) INFO 06-06 18:00:59 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) (EngineCore_DP0 pid=1429615) INFO 06-06 18:00:59 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.47:56829 backend=nccl [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) (EngineCore_DP0 pid=1437257) INFO 06-06 18:01:00 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) (EngineCore_DP0 pid=1098381) INFO 06-06 18:01:00 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=85, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 14x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) (EngineCore_DP0 pid=1437257) INFO 06-06 18:01:00 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) (EngineCore_DP0 pid=1437257) INFO 06-06 18:01:01 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:01:01 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:01:02 [default_loader.py:293] Loading weights took 16.25 seconds (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:01:03 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 17.189895 seconds (pid=1464952, ip=10.128.16.48) ⚙️ Running in WANDB offline mode [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) (EngineCore_DP0 pid=1098387) INFO 06-06 18:01:04 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) (EngineCore_DP0 pid=1098374) INFO 06-06 18:01:04 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.43:41337 backend=nccl [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) (EngineCore_DP0 pid=1098374) INFO 06-06 18:01:04 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) (EngineCore_DP0 pid=1953862) INFO 06-06 18:01:02 [core.py:97] Initializing a V1 LLM engine (vdev) with config: model='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', speculative_config=None, tokenizer='/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=89, served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': True, 'static_all_moe_layers': []} [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) (EngineCore_DP0 pid=1098374) INFO 06-06 18:01:05 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:01:06 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:01:06 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:01:06 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) (EngineCore_DP0 pid=1458576) INFO 06-06 18:01:06 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:01:06 [core.py:278] init engine (profile, create kv cache, warmup model) took 3.65 seconds (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) (EngineCore_DP0 pid=1098374) INFO 06-06 18:01:05 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) WARNING 06-06 18:01:07 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:01:07 [vllm.py:690] Asynchronous scheduling is enabled. (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) WARNING 06-06 18:01:07 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) (EngineCore_DP0 pid=1458544) INFO 06-06 18:01:07 [vllm.py:846] Cudagraph is disabled under eager mode (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) WARNING 06-06 18:01:07 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) (EngineCore_DP0 pid=1246816) INFO 06-06 18:01:07 [default_loader.py:293] Loading weights took 15.69 seconds [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) (EngineCore_DP0 pid=1246816) INFO 06-06 18:01:07 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 16.899541 seconds [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) (EngineCore_DP0 pid=1953862) INFO 06-06 18:01:06 [worker_base.py:289] Injected into for extended collective_rpc calls ['_apply_fp8_weight_loader_patches', '_is_fp8_model', '_quantize_weights_for_fp8', '_restore_param_subclasses', '_undo_param_subclasses', 'begin_weight_update', 'destroy_weights_update_group', 'end_weight_update', 'init_weight_update_communicator', 'load_weights', 'read_named_weights', 'set_numa_affinity', 'test_rpc'] [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) (EngineCore_DP0 pid=1953862) INFO 06-06 18:01:06 [parallel_state.py:1234] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://10.128.16.187:41075 backend=nccl [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) (EngineCore_DP0 pid=1953862) INFO 06-06 18:01:06 [parallel_state.py:1445] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) (EngineCore_DP0 pid=1953862) INFO 06-06 18:01:07 [gpu_model_runner.py:4125] Starting to load model /e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6... [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) (EngineCore_DP0 pid=2131705) INFO 06-06 18:01:10 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) (EngineCore_DP0 pid=2131705) INFO 06-06 18:01:10 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) (EngineCore_DP0 pid=2131705) INFO 06-06 18:01:10 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) (EngineCore_DP0 pid=2131687) INFO 06-06 18:01:11 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) (EngineCore_DP0 pid=2131687) INFO 06-06 18:01:11 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.93 seconds [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) INFO 06-06 18:01:07 [cuda.py:367] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION']. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) (EngineCore_DP0 pid=2131687) WARNING 06-06 18:01:11 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) (EngineCore_DP0 pid=2131687) INFO 06-06 18:01:11 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) (EngineCore_DP0 pid=2131687) WARNING 06-06 18:01:11 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) (EngineCore_DP0 pid=2131687) INFO 06-06 18:01:11 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) WARNING 06-06 18:01:12 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) (EngineCore_DP0 pid=1388596) INFO 06-06 18:01:12 [default_loader.py:293] Loading weights took 12.63 seconds [repeated 9x across cluster] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) (EngineCore_DP0 pid=1388587) INFO 06-06 18:01:13 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 13.482329 seconds [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) (EngineCore_DP0 pid=1388587) INFO 06-06 18:01:15 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) (EngineCore_DP0 pid=1388587) INFO 06-06 18:01:15 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) (EngineCore_DP0 pid=1388587) INFO 06-06 18:01:15 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) (EngineCore_DP0 pid=1455828) INFO 06-06 18:01:16 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 9x across cluster] (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) (EngineCore_DP0 pid=1388591) INFO 06-06 18:01:15 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.79 seconds [repeated 8x across cluster] (FSDPPolicyWorkerBase pid=1464878, ip=10.128.16.48) NCCL version 2.27.7+cuda13.0 (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) (EngineCore_DP0 pid=1455821) WARNING 06-06 18:01:17 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) (EngineCore_DP0 pid=1455821) INFO 06-06 18:01:17 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) (EngineCore_DP0 pid=1455821) WARNING 06-06 18:01:17 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) (EngineCore_DP0 pid=1455821) INFO 06-06 18:01:17 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) WARNING 06-06 18:01:17 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) (EngineCore_DP0 pid=1437265) INFO 06-06 18:01:16 [default_loader.py:293] Loading weights took 13.51 seconds [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) (EngineCore_DP0 pid=1101804) INFO 06-06 18:01:18 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 14.723610 seconds [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) (EngineCore_DP0 pid=1101817) INFO 06-06 18:01:21 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) (EngineCore_DP0 pid=1101817) INFO 06-06 18:01:21 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) (EngineCore_DP0 pid=1101817) INFO 06-06 18:01:21 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) (EngineCore_DP0 pid=1101817) INFO 06-06 18:01:21 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 19x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) (EngineCore_DP0 pid=1101804) INFO 06-06 18:01:21 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.97 seconds [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) (EngineCore_DP0 pid=1101804) WARNING 06-06 18:01:22 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) (EngineCore_DP0 pid=1101804) INFO 06-06 18:01:22 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) (EngineCore_DP0 pid=1101804) WARNING 06-06 18:01:22 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) (EngineCore_DP0 pid=1101804) INFO 06-06 18:01:22 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) WARNING 06-06 18:01:22 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) INFO 06-06 18:01:21 [default_loader.py:293] Loading weights took 13.56 seconds [repeated 8x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] Started monitoring (every 120s) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:01:24] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:01:24] OK: RSS 0.78 GiB | node mem 430.8/858.0 GiB used (50.2%), avail 427.2 GiB (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) INFO 06-06 18:01:21 [gpu_model_runner.py:4222] Model loading took 15.27 GiB memory and 14.683955 seconds [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) (EngineCore_DP0 pid=1953841) INFO 06-06 18:01:24 [gpu_worker.py:373] Available KV cache memory: 49.32 GiB [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) (EngineCore_DP0 pid=1953841) INFO 06-06 18:01:24 [kv_cache_utils.py:1307] GPU KV cache size: 359,104 tokens [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) (EngineCore_DP0 pid=1953841) INFO 06-06 18:01:24 [kv_cache_utils.py:1312] Maximum concurrency for 32,768 tokens per request: 10.96x [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) INFO 06-06 18:01:24 [kernel_warmup.py:44] Skipping FlashInfer autotune because it is disabled. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) INFO 06-06 18:01:24 [core.py:278] init engine (profile, create kv cache, warmup model) took 2.79 seconds [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) WARNING 06-06 18:01:25 [serial_utils.py:57] Allowing insecure serialization using pickle due to VLLM_ALLOW_INSECURE_SERIALIZATION=1 [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) INFO 06-06 18:01:25 [vllm.py:690] Asynchronous scheduling is enabled. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) WARNING 06-06 18:01:25 [vllm.py:735] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) (EngineCore_DP0 pid=1953849) INFO 06-06 18:01:25 [vllm.py:846] Cudagraph is disabled under eager mode [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) WARNING 06-06 18:01:25 [model.py:1350] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] Started monitoring (every 120s) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:01:31] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:01:31] OK: RSS 0.78 GiB | node mem 429.8/858.0 GiB used (50.1%), avail 428.2 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] Started monitoring (every 120s) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:01:40] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:01:40] OK: RSS 0.78 GiB | node mem 387.1/858.0 GiB used (45.1%), avail 470.9 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:01:48] OK: 46 / 131,072 FDs open (0.0% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:01:48] OK: RSS 0.78 GiB | node mem 371.3/858.0 GiB used (43.3%), avail 486.7 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] Started monitoring (every 120s) (FSDPPolicyWorkerBase pid=1452251) INFO 06-06 18:01:51 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) (EngineCore_DP0 pid=1098374) INFO 06-06 18:02:04 [block_pool.py:452] Successfully reset prefix cache (FSDPPolicyWorkerBase pid=1464953, ip=10.128.16.48) INFO 06-06 18:01:52 [pynccl.py:178] pynccl trace buffer enabled: size=10000 dump_dir=/e/data1/datasets/playground/ot-baf/experiments/_nccl_dumps flush_interval=0 s [repeated 7x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:02:08] OK: 104 / 131,072 FDs open (0.1% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:02:08] OK: RSS 1.75 GiB | node mem 213.1/858.0 GiB used (24.8%), avail 644.9 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) (EngineCore_DP0 pid=1101804) INFO 06-06 18:02:04 [block_pool.py:452] Successfully reset prefix cache [repeated 47x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 1512x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 503x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) INFO 06-06 18:02:23 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) INFO 06-06 18:02:23 [loggers.py:259] Engine 000: Avg prompt throughput: 15.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.3%, Prefix cache hit rate: 0.0% (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:02:26 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:02:28 [loggers.py:259] Engine 000: Avg prompt throughput: 75.4 tokens/s, Avg generation throughput: 82.2 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.5%, Prefix cache hit rate: 29.3% [repeated 20x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) INFO 06-06 18:02:32 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:02:33 [loggers.py:259] Engine 000: Avg prompt throughput: 71.2 tokens/s, Avg generation throughput: 62.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.7%, Prefix cache hit rate: 63.3% [repeated 29x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:02:34 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:02:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 91.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.9%, Prefix cache hit rate: 63.3% [repeated 32x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:02:43 [loggers.py:259] Engine 000: Avg prompt throughput: 67.9 tokens/s, Avg generation throughput: 72.3 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.1%, Prefix cache hit rate: 69.6% [repeated 32x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:02:48 [loggers.py:259] Engine 000: Avg prompt throughput: 128.3 tokens/s, Avg generation throughput: 77.4 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.3%, Prefix cache hit rate: 72.0% [repeated 32x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:02:53 [loggers.py:259] Engine 000: Avg prompt throughput: 117.8 tokens/s, Avg generation throughput: 76.6 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 75.1% [repeated 31x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:02:58 [loggers.py:259] Engine 000: Avg prompt throughput: 104.9 tokens/s, Avg generation throughput: 71.6 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.8%, Prefix cache hit rate: 76.8% [repeated 32x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:03:03 [loggers.py:259] Engine 000: Avg prompt throughput: 61.6 tokens/s, Avg generation throughput: 76.2 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.0%, Prefix cache hit rate: 79.4% [repeated 32x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:03:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 81.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.2%, Prefix cache hit rate: 79.4% [repeated 31x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:03:13 [loggers.py:259] Engine 000: Avg prompt throughput: 23.6 tokens/s, Avg generation throughput: 89.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 82.3% [repeated 31x across cluster] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:03:19 [loggers.py:259] Engine 000: Avg prompt throughput: 79.9 tokens/s, Avg generation throughput: 7.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.3%, Prefix cache hit rate: 74.8% [repeated 31x across cluster] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:03:24 [loggers.py:259] Engine 000: Avg prompt throughput: 119.7 tokens/s, Avg generation throughput: 87.2 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.6%, Prefix cache hit rate: 72.3% [repeated 30x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:03:24] OK: 388 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:03:24] OK: RSS 1.12 GiB | node mem 440.0/858.0 GiB used (51.3%), avail 417.9 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) INFO 06-06 18:03:27 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:03:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 87.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.5%, Prefix cache hit rate: 72.3% [repeated 33x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:03:31] OK: 461 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:03:31] OK: RSS 1.15 GiB | node mem 438.9/858.0 GiB used (51.2%), avail 419.1 GiB (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) INFO 06-06 18:03:31 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [repeated 11x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:03:34 [loggers.py:259] Engine 000: Avg prompt throughput: 131.9 tokens/s, Avg generation throughput: 113.3 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 78.4% [repeated 42x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:03:36 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) INFO 06-06 18:03:37 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:03:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 140.1 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.3%, Prefix cache hit rate: 78.4% [repeated 46x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:03:40] OK: 504 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:03:40] OK: RSS 1.15 GiB | node mem 396.4/858.0 GiB used (46.2%), avail 461.6 GiB (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) INFO 06-06 18:03:37 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:03:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 140.0 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.4%, Prefix cache hit rate: 78.4% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:03:48] OK: 247 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:03:48] OK: RSS 1.13 GiB | node mem 380.1/858.0 GiB used (44.3%), avail 477.9 GiB (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:03:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 140.2 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 78.4% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:03:54 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 139.3 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.4%, Prefix cache hit rate: 78.4% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:03:59 [loggers.py:259] Engine 000: Avg prompt throughput: 108.2 tokens/s, Avg generation throughput: 80.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.1%, Prefix cache hit rate: 81.8% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:04 [loggers.py:259] Engine 000: Avg prompt throughput: 28.3 tokens/s, Avg generation throughput: 76.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.2%, Prefix cache hit rate: 85.2% [repeated 47x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:04:08] OK: 301 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:04:08] OK: RSS 1.77 GiB | node mem 213.3/858.0 GiB used (24.9%), avail 644.6 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:09 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 95.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.4%, Prefix cache hit rate: 85.2% [repeated 47x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 95.8 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.5%, Prefix cache hit rate: 85.2% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 96.0 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.6%, Prefix cache hit rate: 85.2% [repeated 45x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 95.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.8%, Prefix cache hit rate: 85.2% [repeated 46x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 45x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 96.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.9%, Prefix cache hit rate: 85.2% [repeated 45x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) INFO 06-06 18:04:32 [hf.py:318] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 96.0 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.1%, Prefix cache hit rate: 85.2% [repeated 46x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 96.0 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.2%, Prefix cache hit rate: 85.2% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 96.0 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.3%, Prefix cache hit rate: 85.2% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 96.2 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.5%, Prefix cache hit rate: 85.2% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:54 [loggers.py:259] Engine 000: Avg prompt throughput: 87.3 tokens/s, Avg generation throughput: 132.1 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.8%, Prefix cache hit rate: 84.2% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:04:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 144.6 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.0%, Prefix cache hit rate: 84.2% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:05:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 144.2 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.2%, Prefix cache hit rate: 84.2% [repeated 47x across cluster] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:05:09 [loggers.py:259] Engine 000: Avg prompt throughput: 53.6 tokens/s, Avg generation throughput: 116.9 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 86.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:05:14 [loggers.py:259] Engine 000: Avg prompt throughput: 132.9 tokens/s, Avg generation throughput: 110.6 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.7%, Prefix cache hit rate: 87.5% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:05:19 [loggers.py:259] Engine 000: Avg prompt throughput: 310.2 tokens/s, Avg generation throughput: 193.9 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.9%, Prefix cache hit rate: 79.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:05:24] OK: 417 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:05:24] OK: RSS 1.13 GiB | node mem 440.2/858.0 GiB used (51.3%), avail 417.8 GiB (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:05:24 [loggers.py:259] Engine 000: Avg prompt throughput: 26.1 tokens/s, Avg generation throughput: 227.9 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 6.5%, Prefix cache hit rate: 80.5% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:05:30 [loggers.py:259] Engine 000: Avg prompt throughput: 151.1 tokens/s, Avg generation throughput: 103.2 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.1%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:05:31] OK: 429 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:05:31] OK: RSS 1.19 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 419.0 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:05:35 [loggers.py:259] Engine 000: Avg prompt throughput: 90.0 tokens/s, Avg generation throughput: 120.0 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.4%, Prefix cache hit rate: 89.0% [repeated 47x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:05:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 172.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.6%, Prefix cache hit rate: 89.0% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:05:40] OK: 464 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:05:40] OK: RSS 1.16 GiB | node mem 396.5/858.0 GiB used (46.2%), avail 461.4 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:05:45 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 129.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.6%, Prefix cache hit rate: 89.0% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:05:48] OK: 437 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:05:48] OK: RSS 1.17 GiB | node mem 380.2/858.0 GiB used (44.3%), avail 477.8 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:05:50 [loggers.py:259] Engine 000: Avg prompt throughput: 217.3 tokens/s, Avg generation throughput: 81.2 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.0%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:05:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 130.0 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.2%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:06:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 94.9 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.9%, Prefix cache hit rate: 88.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:06:05 [loggers.py:259] Engine 000: Avg prompt throughput: 98.5 tokens/s, Avg generation throughput: 118.4 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.6%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:06:08] OK: 419 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:06:08] OK: RSS 1.79 GiB | node mem 213.4/858.0 GiB used (24.9%), avail 644.5 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:06:10 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 129.9 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.8%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:06:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 129.8 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.9%, Prefix cache hit rate: 88.3% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 53x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) INFO 06-06 18:06:20 [loggers.py:259] Engine 000: Avg prompt throughput: 441.6 tokens/s, Avg generation throughput: 308.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.4%, Prefix cache hit rate: 86.3% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) INFO 06-06 18:06:25 [loggers.py:259] Engine 000: Avg prompt throughput: 56.8 tokens/s, Avg generation throughput: 302.4 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.8%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:06:30 [loggers.py:259] Engine 000: Avg prompt throughput: 190.9 tokens/s, Avg generation throughput: 224.2 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.1%, Prefix cache hit rate: 91.1% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 56x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) INFO 06-06 18:06:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 129.3 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 5.9%, Prefix cache hit rate: 88.1% [repeated 47x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:06:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 238.3 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.8%, Prefix cache hit rate: 91.2% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 38x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:06:45 [loggers.py:259] Engine 000: Avg prompt throughput: 117.8 tokens/s, Avg generation throughput: 218.2 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.3%, Prefix cache hit rate: 91.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:06:50 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 194.2 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.2%, Prefix cache hit rate: 91.2% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:06:55 [loggers.py:259] Engine 000: Avg prompt throughput: 318.9 tokens/s, Avg generation throughput: 222.8 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.3%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:07:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 278.9 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.9%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:07:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 239.7 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.3%, Prefix cache hit rate: 90.9% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:07:10 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 238.4 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.6%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:07:15 [loggers.py:259] Engine 000: Avg prompt throughput: 206.5 tokens/s, Avg generation throughput: 245.4 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.9%, Prefix cache hit rate: 90.8% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:07:20 [loggers.py:259] Engine 000: Avg prompt throughput: 27.3 tokens/s, Avg generation throughput: 261.2 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.3%, Prefix cache hit rate: 90.8% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:07:24] OK: 290 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:07:24] OK: RSS 1.15 GiB | node mem 440.2/858.0 GiB used (51.3%), avail 417.8 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) INFO 06-06 18:07:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 311.1 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.4%, Prefix cache hit rate: 87.3% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) INFO 06-06 18:07:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 293.1 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.3%, Prefix cache hit rate: 87.3% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:07:31] OK: 318 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:07:31] OK: RSS 1.20 GiB | node mem 439.1/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) INFO 06-06 18:07:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 327.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.4%, Prefix cache hit rate: 90.7% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) INFO 06-06 18:07:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 279.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 15.9%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:07:40] OK: 387 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:07:40] OK: RSS 1.17 GiB | node mem 396.6/858.0 GiB used (46.2%), avail 461.4 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) INFO 06-06 18:07:45 [loggers.py:259] Engine 000: Avg prompt throughput: 79.4 tokens/s, Avg generation throughput: 244.1 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.3%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:07:48] OK: 369 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:07:48] OK: RSS 1.17 GiB | node mem 380.2/858.0 GiB used (44.3%), avail 477.8 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) INFO 06-06 18:07:50 [loggers.py:259] Engine 000: Avg prompt throughput: 111.6 tokens/s, Avg generation throughput: 269.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.8%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) INFO 06-06 18:07:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 303.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.2%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:08:00 [loggers.py:259] Engine 000: Avg prompt throughput: 131.2 tokens/s, Avg generation throughput: 469.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.2%, Prefix cache hit rate: 86.2% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:08:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 442.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.8%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:08:08] OK: 417 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:08:08] OK: RSS 1.80 GiB | node mem 213.5/858.0 GiB used (24.9%), avail 644.4 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:08:10 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 444.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.4%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:08:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 439.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.7%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:08:20 [loggers.py:259] Engine 000: Avg prompt throughput: 295.8 tokens/s, Avg generation throughput: 405.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.5%, Prefix cache hit rate: 85.5% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:08:25 [loggers.py:259] Engine 000: Avg prompt throughput: 28.4 tokens/s, Avg generation throughput: 405.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.2%, Prefix cache hit rate: 85.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) INFO 06-06 18:08:30 [loggers.py:259] Engine 000: Avg prompt throughput: 44.4 tokens/s, Avg generation throughput: 124.5 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 7.3%, Prefix cache hit rate: 86.4% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) INFO 06-06 18:08:35 [loggers.py:259] Engine 000: Avg prompt throughput: 85.0 tokens/s, Avg generation throughput: 154.9 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 7.6%, Prefix cache hit rate: 86.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:08:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 220.6 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.7%, Prefix cache hit rate: 88.0% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:08:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 220.4 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.0%, Prefix cache hit rate: 88.0% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:08:51 [loggers.py:259] Engine 000: Avg prompt throughput: 58.7 tokens/s, Avg generation throughput: 252.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.1%, Prefix cache hit rate: 88.1% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:08:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 230.9 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.6%, Prefix cache hit rate: 88.1% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 220.6 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.9%, Prefix cache hit rate: 88.1% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 220.3 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.2%, Prefix cache hit rate: 88.1% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:11 [loggers.py:259] Engine 000: Avg prompt throughput: 74.1 tokens/s, Avg generation throughput: 253.3 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.7%, Prefix cache hit rate: 87.9% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 265.3 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.0%, Prefix cache hit rate: 87.9% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:21 [loggers.py:259] Engine 000: Avg prompt throughput: 82.6 tokens/s, Avg generation throughput: 307.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.6%, Prefix cache hit rate: 87.7% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:09:24] OK: 292 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:09:24] OK: RSS 1.19 GiB | node mem 440.2/858.0 GiB used (51.3%), avail 417.8 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:26 [loggers.py:259] Engine 000: Avg prompt throughput: 34.3 tokens/s, Avg generation throughput: 295.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.0%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 308.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.5%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:09:31] OK: 328 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:09:31] OK: RSS 1.21 GiB | node mem 439.1/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 308.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.9%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:09:40] OK: 387 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:09:40] OK: RSS 1.21 GiB | node mem 396.6/858.0 GiB used (46.2%), avail 461.4 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 299.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.9%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:46 [loggers.py:259] Engine 000: Avg prompt throughput: 102.8 tokens/s, Avg generation throughput: 298.2 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.9%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:09:48] OK: 390 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:09:48] OK: RSS 1.21 GiB | node mem 380.2/858.0 GiB used (44.3%), avail 477.7 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 305.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.3%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:09:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 303.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.7%, Prefix cache hit rate: 87.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:10:01 [loggers.py:259] Engine 000: Avg prompt throughput: 140.8 tokens/s, Avg generation throughput: 257.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.5%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:10:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 257.9 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.9%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:10:08] OK: 463 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:10:08] OK: RSS 1.81 GiB | node mem 213.6/858.0 GiB used (24.9%), avail 644.4 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:10:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 258.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.3%, Prefix cache hit rate: 87.5% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 425.3 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.4%, Prefix cache hit rate: 90.1% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 425.0 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.0%, Prefix cache hit rate: 90.1% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 424.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.6%, Prefix cache hit rate: 90.1% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:31 [loggers.py:259] Engine 000: Avg prompt throughput: 272.4 tokens/s, Avg generation throughput: 471.3 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.0%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:36 [loggers.py:259] Engine 000: Avg prompt throughput: 69.4 tokens/s, Avg generation throughput: 486.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.0%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 461.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.9%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:46 [loggers.py:259] Engine 000: Avg prompt throughput: 59.4 tokens/s, Avg generation throughput: 447.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.4%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 469.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.0%, Prefix cache hit rate: 90.2% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:10:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 469.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.7%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:11:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 468.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.3%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:11:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 469.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.0%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) INFO 06-06 18:11:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 467.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.6%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.1 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.8%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.2%, Prefix cache hit rate: 86.2% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:11:24] OK: 261 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:11:24] OK: RSS 1.19 GiB | node mem 440.2/858.0 GiB used (51.3%), avail 417.8 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.2 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.7%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:11:31] OK: 329 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:11:31] OK: RSS 1.21 GiB | node mem 439.1/858.0 GiB used (51.2%), avail 418.9 GiB (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:31 [loggers.py:259] Engine 000: Avg prompt throughput: 45.6 tokens/s, Avg generation throughput: 357.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.5%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 388.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.0%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:11:40] OK: 353 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:11:40] OK: RSS 1.21 GiB | node mem 396.6/858.0 GiB used (46.2%), avail 461.3 GiB (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 390.0 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.6%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 387.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.1%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:11:47 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:11:48] OK: 388 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:11:48] OK: RSS 1.21 GiB | node mem 380.3/858.0 GiB used (44.3%), avail 477.7 GiB (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 388.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.6%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:11:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 388.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.2%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:12:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 388.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.7%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:12:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 370.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.5%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (skyrl_entrypoint pid=1452083) [fd-monitor] [18:12:08] OK: 466 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:12:08] OK: RSS 1.81 GiB | node mem 213.6/858.0 GiB used (24.9%), avail 644.3 GiB (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:12:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 338.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.0%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) INFO 06-06 18:12:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 338.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.4%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) INFO 06-06 18:12:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 279.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.5%, Prefix cache hit rate: 88.8% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) INFO 06-06 18:12:26 [loggers.py:259] Engine 000: Avg prompt throughput: 134.9 tokens/s, Avg generation throughput: 344.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.2%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) INFO 06-06 18:12:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 349.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.5%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:12:35 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) INFO 06-06 18:12:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 323.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.0%, Prefix cache hit rate: 87.4% [repeated 50x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) INFO 06-06 18:12:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 324.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.5%, Prefix cache hit rate: 87.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) INFO 06-06 18:12:47 [loggers.py:259] Engine 000: Avg prompt throughput: 39.7 tokens/s, Avg generation throughput: 339.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.8%, Prefix cache hit rate: 87.4% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) INFO 06-06 18:12:52 [loggers.py:259] Engine 000: Avg prompt throughput: 69.0 tokens/s, Avg generation throughput: 363.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.8%, Prefix cache hit rate: 87.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:12:54 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) INFO 06-06 18:12:57 [loggers.py:259] Engine 000: Avg prompt throughput: 47.3 tokens/s, Avg generation throughput: 420.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.6%, Prefix cache hit rate: 86.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) INFO 06-06 18:13:02 [loggers.py:259] Engine 000: Avg prompt throughput: 128.2 tokens/s, Avg generation throughput: 425.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.2%, Prefix cache hit rate: 86.9% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:06 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) INFO 06-06 18:13:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 431.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.8%, Prefix cache hit rate: 86.9% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:13:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 186.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.0%, Prefix cache hit rate: 85.4% [repeated 51x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:15 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:13:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 186.6 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.2%, Prefix cache hit rate: 85.4% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:13:18 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:13:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 163.1 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.5%, Prefix cache hit rate: 85.4% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:13:24] OK: 364 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:13:24] OK: RSS 1.19 GiB | node mem 440.2/858.0 GiB used (51.3%), avail 417.8 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:13:27 [loggers.py:259] Engine 000: Avg prompt throughput: 266.9 tokens/s, Avg generation throughput: 184.1 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.1%, Prefix cache hit rate: 84.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:13:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:13:31] OK: 371 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:13:31] OK: RSS 1.22 GiB | node mem 439.1/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:13:32 [loggers.py:259] Engine 000: Avg prompt throughput: 21.6 tokens/s, Avg generation throughput: 214.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.5%, Prefix cache hit rate: 86.4% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:13:37 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:13:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 280.3 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.9%, Prefix cache hit rate: 86.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:13:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:13:40] OK: 342 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:13:40] OK: RSS 1.23 GiB | node mem 396.6/858.0 GiB used (46.2%), avail 461.3 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:13:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:13:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 250.3 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.0%, Prefix cache hit rate: 84.1% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:13:43 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:13:45 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:13:47 [loggers.py:259] Engine 000: Avg prompt throughput: 35.1 tokens/s, Avg generation throughput: 263.8 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.2%, Prefix cache hit rate: 84.2% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:13:48] OK: 392 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:13:48] OK: RSS 1.22 GiB | node mem 380.3/858.0 GiB used (44.3%), avail 477.7 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:13:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:13:52 [loggers.py:259] Engine 000: Avg prompt throughput: 311.6 tokens/s, Avg generation throughput: 361.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.7%, Prefix cache hit rate: 83.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:13:51 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) INFO 06-06 18:13:57 [loggers.py:259] Engine 000: Avg prompt throughput: 41.3 tokens/s, Avg generation throughput: 368.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.9%, Prefix cache hit rate: 83.9% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) ERROR 06-06 18:13:58 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:13:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:14:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 140.1 tokens/s, Running: 3 reqs, Waiting: 0 reqs, GPU KV cache usage: 12.8%, Prefix cache hit rate: 86.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) ERROR 06-06 18:14:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:14:07 [loggers.py:259] Engine 000: Avg prompt throughput: 45.1 tokens/s, Avg generation throughput: 145.6 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.3%, Prefix cache hit rate: 86.4% [repeated 48x across cluster] (skyrl_entrypoint pid=1452083) [fd-monitor] [18:14:08] OK: 542 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:14:08] OK: RSS 1.83 GiB | node mem 213.7/858.0 GiB used (24.9%), avail 644.3 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:14:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 187.6 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.6%, Prefix cache hit rate: 86.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:14:13 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:14:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 187.5 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 13.8%, Prefix cache hit rate: 86.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:14:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 186.8 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.1%, Prefix cache hit rate: 86.4% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:14:24 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) INFO 06-06 18:14:27 [loggers.py:259] Engine 000: Avg prompt throughput: 209.5 tokens/s, Avg generation throughput: 624.1 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.5%, Prefix cache hit rate: 89.5% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:14:30 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:14:32 [loggers.py:259] Engine 000: Avg prompt throughput: 71.3 tokens/s, Avg generation throughput: 205.6 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.7%, Prefix cache hit rate: 86.2% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) INFO 06-06 18:14:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 225.3 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 14.9%, Prefix cache hit rate: 86.2% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:14:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) INFO 06-06 18:14:42 [loggers.py:259] Engine 000: Avg prompt throughput: 70.2 tokens/s, Avg generation throughput: 543.9 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.9%, Prefix cache hit rate: 89.9% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:14:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) INFO 06-06 18:14:47 [loggers.py:259] Engine 000: Avg prompt throughput: 22.7 tokens/s, Avg generation throughput: 571.1 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.5%, Prefix cache hit rate: 90.1% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] Error in preprocessing prompt inputs [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] Traceback (most recent call last): [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] return await asyncio.gather( [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] raise VLLMValidationError( [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:14:46 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) INFO 06-06 18:14:52 [loggers.py:259] Engine 000: Avg prompt throughput: 23.0 tokens/s, Avg generation throughput: 516.5 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.7%, Prefix cache hit rate: 90.2% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 35x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) INFO 06-06 18:14:57 [loggers.py:259] Engine 000: Avg prompt throughput: 21.6 tokens/s, Avg generation throughput: 532.8 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.0%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:14:58 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) INFO 06-06 18:15:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 387.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.1%, Prefix cache hit rate: 87.8% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:01 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) INFO 06-06 18:15:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 370.0 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.7%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:06 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) INFO 06-06 18:15:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 322.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.2%, Prefix cache hit rate: 87.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:18 [loggers.py:259] Engine 000: Avg prompt throughput: 24.2 tokens/s, Avg generation throughput: 389.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.8%, Prefix cache hit rate: 85.0% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 364.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.4%, Prefix cache hit rate: 85.0% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:15:24] OK: 411 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:15:24] OK: RSS 1.20 GiB | node mem 440.3/858.0 GiB used (51.3%), avail 417.7 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:28 [loggers.py:259] Engine 000: Avg prompt throughput: 125.4 tokens/s, Avg generation throughput: 423.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.5%, Prefix cache hit rate: 85.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:15:31 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:15:31] OK: 428 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:15:31] OK: RSS 1.23 GiB | node mem 439.1/858.0 GiB used (51.2%), avail 418.9 GiB (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 448.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.9%, Prefix cache hit rate: 85.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:15:34 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 399.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.0%, Prefix cache hit rate: 85.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:15:40] OK: 332 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:15:40] OK: RSS 1.24 GiB | node mem 396.7/858.0 GiB used (46.2%), avail 461.3 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:15:36 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:43 [loggers.py:259] Engine 000: Avg prompt throughput: 44.6 tokens/s, Avg generation throughput: 395.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.8%, Prefix cache hit rate: 85.8% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 417.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.4%, Prefix cache hit rate: 85.8% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:15:48] OK: 387 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:15:48] OK: RSS 1.22 GiB | node mem 380.4/858.0 GiB used (44.3%), avail 477.6 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) ERROR 06-06 18:15:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:53 [loggers.py:259] Engine 000: Avg prompt throughput: 37.5 tokens/s, Avg generation throughput: 371.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.1%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:15:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 336.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.9%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:15:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:16:02 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:16:03 [loggers.py:259] Engine 000: Avg prompt throughput: 84.2 tokens/s, Avg generation throughput: 336.0 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.8%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 35x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:05 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (skyrl_entrypoint pid=1452083) [fd-monitor] [18:16:08] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:16:08] OK: RSS 1.84 GiB | node mem 213.7/858.0 GiB used (24.9%), avail 644.2 GiB (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:16:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 372.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.8%, Prefix cache hit rate: 86.0% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 7x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 7x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:10 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:16:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:16:13 [loggers.py:259] Engine 000: Avg prompt throughput: 84.4 tokens/s, Avg generation throughput: 332.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.8%, Prefix cache hit rate: 86.3% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:16:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:16:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:16:18 [loggers.py:259] Engine 000: Avg prompt throughput: 51.0 tokens/s, Avg generation throughput: 372.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.4%, Prefix cache hit rate: 86.3% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) ERROR 06-06 18:16:22 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:16:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 377.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.3%, Prefix cache hit rate: 86.7% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 7x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 7x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:16:26 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:16:28 [loggers.py:259] Engine 000: Avg prompt throughput: 21.9 tokens/s, Avg generation throughput: 422.0 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.6%, Prefix cache hit rate: 86.8% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:16:28 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:16:33 [loggers.py:259] Engine 000: Avg prompt throughput: 9.6 tokens/s, Avg generation throughput: 377.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.7%, Prefix cache hit rate: 86.9% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:16:38 [loggers.py:259] Engine 000: Avg prompt throughput: 207.8 tokens/s, Avg generation throughput: 390.7 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 15.6%, Prefix cache hit rate: 86.7% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 63x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:16:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:16:43 [loggers.py:259] Engine 000: Avg prompt throughput: 65.1 tokens/s, Avg generation throughput: 381.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.1%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) ERROR 06-06 18:16:42 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 9x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 9x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:16:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 423.8 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.7%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:16:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:16:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 422.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.1%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:16:52 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) INFO 06-06 18:16:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 367.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.4%, Prefix cache hit rate: 86.9% [repeated 47x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:17:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:17:03 [loggers.py:259] Engine 000: Avg prompt throughput: 148.2 tokens/s, Avg generation throughput: 358.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.0%, Prefix cache hit rate: 86.9% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:17:08 [loggers.py:259] Engine 000: Avg prompt throughput: 278.3 tokens/s, Avg generation throughput: 411.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.8%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:17:10 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:17:13 [loggers.py:259] Engine 000: Avg prompt throughput: 163.2 tokens/s, Avg generation throughput: 424.3 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.8%, Prefix cache hit rate: 86.9% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:17 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:17:18 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) INFO 06-06 18:17:18 [loggers.py:259] Engine 000: Avg prompt throughput: 68.7 tokens/s, Avg generation throughput: 460.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.0%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:17:20 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:17:23 [loggers.py:259] Engine 000: Avg prompt throughput: 70.1 tokens/s, Avg generation throughput: 703.0 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.4%, Prefix cache hit rate: 91.9% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:17:24] OK: 362 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:17:24] OK: RSS 1.20 GiB | node mem 440.2/858.0 GiB used (51.3%), avail 417.7 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:17:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 718.4 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.4%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 45x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) ERROR 06-06 18:17:30 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:17:31] OK: 365 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:17:31] OK: RSS 1.23 GiB | node mem 439.1/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:17:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 711.4 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 58.2%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:17:31 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:17:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 675.3 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.2%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:17:39 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:17:41] OK: 341 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:17:41] OK: RSS 1.26 GiB | node mem 396.7/858.0 GiB used (46.2%), avail 461.2 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:17:43 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 674.9 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.1%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:17:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:17:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 674.8 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.1%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:17:48] OK: 384 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:17:48] OK: RSS 1.24 GiB | node mem 380.4/858.0 GiB used (44.3%), avail 477.6 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:17:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:17:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 675.1 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 62.0%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 25x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387585, ip=10.128.16.37) ERROR 06-06 18:17:55 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:17:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 645.7 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 62.2%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 545.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.3%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:18:08] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:18:08] OK: RSS 1.85 GiB | node mem 213.8/858.0 GiB used (24.9%), avail 644.2 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 543.3 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.1%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) ERROR 06-06 18:18:09 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:13 [loggers.py:259] Engine 000: Avg prompt throughput: 178.8 tokens/s, Avg generation throughput: 549.8 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 63.5%, Prefix cache hit rate: 91.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 586.5 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 62.6%, Prefix cache hit rate: 91.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:23 [loggers.py:259] Engine 000: Avg prompt throughput: 85.8 tokens/s, Avg generation throughput: 572.1 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.3%, Prefix cache hit rate: 91.8% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:18:27 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 586.9 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.1%, Prefix cache hit rate: 91.8% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:18:30 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:33 [loggers.py:259] Engine 000: Avg prompt throughput: 37.8 tokens/s, Avg generation throughput: 631.0 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 67.2%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) ERROR 06-06 18:18:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:38 [loggers.py:259] Engine 000: Avg prompt throughput: 48.0 tokens/s, Avg generation throughput: 628.2 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 68.8%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:18:39 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:43 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 621.1 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 68.2%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:18:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) INFO 06-06 18:18:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 586.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 69.1%, Prefix cache hit rate: 91.9% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:18:44 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 53x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:18:54 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 692.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.2%, Prefix cache hit rate: 89.6% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:18:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 614.9 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.0%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:19:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 613.2 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.3%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:19:09 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 564.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.2%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:19:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 480.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.5%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:19:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 476.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.2%, Prefix cache hit rate: 89.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:19:24 [loggers.py:259] Engine 000: Avg prompt throughput: 146.5 tokens/s, Avg generation throughput: 534.3 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.1%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:19:24] OK: 360 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:19:24] OK: RSS 1.21 GiB | node mem 440.3/858.0 GiB used (51.3%), avail 417.7 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:19:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 537.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.8%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:19:31] OK: 367 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:19:31] OK: RSS 1.24 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 419.0 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:19:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 476.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.5%, Prefix cache hit rate: 89.8% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:19:38 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) INFO 06-06 18:19:39 [loggers.py:259] Engine 000: Avg prompt throughput: 157.8 tokens/s, Avg generation throughput: 443.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.4%, Prefix cache hit rate: 87.6% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:19:41] OK: 331 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:19:41] OK: RSS 1.27 GiB | node mem 396.7/858.0 GiB used (46.2%), avail 461.3 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:19:38 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1953615, ip=10.128.16.187) INFO 06-06 18:19:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 462.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.4%, Prefix cache hit rate: 89.8% [repeated 47x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 35x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:19:48] OK: 388 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:19:48] OK: RSS 1.24 GiB | node mem 380.4/858.0 GiB used (44.3%), avail 477.5 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) INFO 06-06 18:19:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 469.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.2%, Prefix cache hit rate: 87.6% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:19:54 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 393.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.9%, Prefix cache hit rate: 90.0% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:19:56 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:19:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 391.0 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.5%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:20:04 [loggers.py:259] Engine 000: Avg prompt throughput: 241.7 tokens/s, Avg generation throughput: 410.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.9%, Prefix cache hit rate: 90.1% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:20:04 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:20:08] OK: 565 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:20:08] OK: RSS 1.85 GiB | node mem 213.8/858.0 GiB used (24.9%), avail 644.2 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:20:09 [loggers.py:259] Engine 000: Avg prompt throughput: 120.7 tokens/s, Avg generation throughput: 512.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.8%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) ERROR 06-06 18:20:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:20:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 497.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.8%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:20:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:20:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 472.8 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.4%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:20:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 473.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.1%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:20:29 [loggers.py:259] Engine 000: Avg prompt throughput: 21.0 tokens/s, Avg generation throughput: 512.1 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.8%, Prefix cache hit rate: 90.1% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:20:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 463.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.3%, Prefix cache hit rate: 90.1% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455592, ip=10.128.16.46) INFO 06-06 18:20:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 428.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.9%, Prefix cache hit rate: 90.1% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:20:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 228.6 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.5%, Prefix cache hit rate: 90.0% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) INFO 06-06 18:20:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 229.2 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.8%, Prefix cache hit rate: 90.0% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:20:53 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) INFO 06-06 18:20:54 [loggers.py:259] Engine 000: Avg prompt throughput: 124.9 tokens/s, Avg generation throughput: 415.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.4%, Prefix cache hit rate: 86.7% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) INFO 06-06 18:20:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 363.1 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.9%, Prefix cache hit rate: 86.7% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) INFO 06-06 18:21:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 326.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.4%, Prefix cache hit rate: 86.7% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) INFO 06-06 18:21:09 [loggers.py:259] Engine 000: Avg prompt throughput: 99.2 tokens/s, Avg generation throughput: 343.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.4%, Prefix cache hit rate: 86.7% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) INFO 06-06 18:21:14 [loggers.py:259] Engine 000: Avg prompt throughput: 38.6 tokens/s, Avg generation throughput: 396.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.9%, Prefix cache hit rate: 86.7% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) INFO 06-06 18:21:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 337.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.7%, Prefix cache hit rate: 86.7% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:21:24] OK: 386 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:21:24] OK: RSS 1.21 GiB | node mem 440.2/858.0 GiB used (51.3%), avail 417.7 GiB (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) INFO 06-06 18:21:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 215.5 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.4%, Prefix cache hit rate: 89.4% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) INFO 06-06 18:21:30 [loggers.py:259] Engine 000: Avg prompt throughput: 245.2 tokens/s, Avg generation throughput: 217.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.2%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:21:32] OK: 365 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:21:32] OK: RSS 1.24 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) INFO 06-06 18:21:35 [loggers.py:259] Engine 000: Avg prompt throughput: 11.4 tokens/s, Avg generation throughput: 274.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.7%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) INFO 06-06 18:21:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 302.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.1%, Prefix cache hit rate: 89.3% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:21:41] OK: 332 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:21:41] OK: RSS 1.27 GiB | node mem 396.8/858.0 GiB used (46.2%), avail 461.2 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) INFO 06-06 18:21:45 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 261.4 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.3%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:21:46 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 66x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:21:48] OK: 367 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:21:48] OK: RSS 1.28 GiB | node mem 380.5/858.0 GiB used (44.3%), avail 477.5 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429271, ip=10.128.16.41) INFO 06-06 18:21:50 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 246.3 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.9%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:21:47 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 32x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:21:55 [loggers.py:259] Engine 000: Avg prompt throughput: 42.0 tokens/s, Avg generation throughput: 593.7 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.7%, Prefix cache hit rate: 91.6% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:22:00 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 559.7 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.2%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 35x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 558.1 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.0%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 51x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:22:08] OK: 565 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:22:08] OK: RSS 1.86 GiB | node mem 213.8/858.0 GiB used (24.9%), avail 644.2 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:10 [loggers.py:259] Engine 000: Avg prompt throughput: 114.3 tokens/s, Avg generation throughput: 585.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 62.4%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 41x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:15 [loggers.py:259] Engine 000: Avg prompt throughput: 33.4 tokens/s, Avg generation throughput: 646.4 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.6%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 49x across cluster] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:22:13 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:20 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 648.2 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.5%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1429233, ip=10.128.16.47) ERROR 06-06 18:22:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 606.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 63.1%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:30 [loggers.py:259] Engine 000: Avg prompt throughput: 8.8 tokens/s, Avg generation throughput: 585.4 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.0%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:22:31 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:35 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 603.3 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.8%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:22:31 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 603.8 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.7%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 34x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:45 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 603.5 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 66.5%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 39x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:50 [loggers.py:259] Engine 000: Avg prompt throughput: 20.3 tokens/s, Avg generation throughput: 642.7 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 69.2%, Prefix cache hit rate: 91.7% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 24x across cluster] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:22:53 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:22:55 [loggers.py:259] Engine 000: Avg prompt throughput: 90.1 tokens/s, Avg generation throughput: 654.9 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 71.3%, Prefix cache hit rate: 91.7% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:23:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 649.1 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 72.2%, Prefix cache hit rate: 91.7% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:23:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 617.9 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 71.5%, Prefix cache hit rate: 91.7% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:23:08 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:23:10 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 576.2 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 70.1%, Prefix cache hit rate: 91.7% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 81x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:23:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 558.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 70.9%, Prefix cache hit rate: 91.7% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1246563, ip=10.128.16.45) ERROR 06-06 18:23:16 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 53x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) INFO 06-06 18:23:20 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 558.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 71.7%, Prefix cache hit rate: 91.7% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429367, ip=10.128.16.47) ERROR 06-06 18:23:20 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:23:24] OK: 363 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:23:24] OK: RSS 1.22 GiB | node mem 440.4/858.0 GiB used (51.3%), avail 417.6 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) INFO 06-06 18:23:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 439.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.4%, Prefix cache hit rate: 90.0% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:23:23 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 24x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) INFO 06-06 18:23:30 [loggers.py:259] Engine 000: Avg prompt throughput: 273.6 tokens/s, Avg generation throughput: 332.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.1%, Prefix cache hit rate: 89.9% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:23:28 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:23:32] OK: 396 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:23:32] OK: RSS 1.25 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) INFO 06-06 18:23:35 [loggers.py:259] Engine 000: Avg prompt throughput: 149.8 tokens/s, Avg generation throughput: 452.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.1%, Prefix cache hit rate: 88.8% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:23:36 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) INFO 06-06 18:23:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 457.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 37.9%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:23:41] OK: 512 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:23:41] OK: RSS 1.28 GiB | node mem 396.8/858.0 GiB used (46.2%), avail 461.2 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:23:39 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:23:45 [loggers.py:259] Engine 000: Avg prompt throughput: 53.1 tokens/s, Avg generation throughput: 725.5 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 78.8%, Prefix cache hit rate: 88.0% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:23:49] OK: 378 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:23:49] OK: RSS 1.28 GiB | node mem 380.5/858.0 GiB used (44.4%), avail 477.4 GiB (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:23:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:23:50 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 764.4 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 79.9%, Prefix cache hit rate: 88.0% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:23:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:23:55 [loggers.py:259] Engine 000: Avg prompt throughput: 81.3 tokens/s, Avg generation throughput: 746.8 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 81.4%, Prefix cache hit rate: 88.0% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:24:00 [loggers.py:259] Engine 000: Avg prompt throughput: 78.9 tokens/s, Avg generation throughput: 790.6 tokens/s, Running: 22 reqs, Waiting: 0 reqs, GPU KV cache usage: 82.8%, Prefix cache hit rate: 88.0% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:24:05 [loggers.py:259] Engine 000: Avg prompt throughput: 3.2 tokens/s, Avg generation throughput: 779.2 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 82.0%, Prefix cache hit rate: 88.1% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:24:08] OK: 565 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:24:08] OK: RSS 1.87 GiB | node mem 213.8/858.0 GiB used (24.9%), avail 644.2 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:24:10 [loggers.py:259] Engine 000: Avg prompt throughput: 26.7 tokens/s, Avg generation throughput: 774.7 tokens/s, Running: 22 reqs, Waiting: 0 reqs, GPU KV cache usage: 83.9%, Prefix cache hit rate: 88.1% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:24:11 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:24:15 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 791.8 tokens/s, Running: 22 reqs, Waiting: 0 reqs, GPU KV cache usage: 85.0%, Prefix cache hit rate: 88.1% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) ERROR 06-06 18:24:15 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1953486, ip=10.128.16.187) ERROR 06-06 18:24:16 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:24:20 [loggers.py:259] Engine 000: Avg prompt throughput: 138.7 tokens/s, Avg generation throughput: 836.1 tokens/s, Running: 24 reqs, Waiting: 0 reqs, GPU KV cache usage: 89.2%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:24:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:24:21 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:24:25 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 845.3 tokens/s, Running: 22 reqs, Waiting: 0 reqs, GPU KV cache usage: 87.2%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387586, ip=10.128.16.37) ERROR 06-06 18:24:22 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388481, ip=10.128.16.44) INFO 06-06 18:24:30 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 794.2 tokens/s, Running: 21 reqs, Waiting: 0 reqs, GPU KV cache usage: 87.3%, Prefix cache hit rate: 88.2% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) INFO 06-06 18:24:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 378.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.4%, Prefix cache hit rate: 90.4% [repeated 49x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:24:36 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) INFO 06-06 18:24:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 372.7 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 38.9%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:24:38 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 38x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1101518, ip=10.128.16.38) INFO 06-06 18:24:46 [loggers.py:259] Engine 000: Avg prompt throughput: 203.9 tokens/s, Avg generation throughput: 428.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.5%, Prefix cache hit rate: 90.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 45x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) ERROR 06-06 18:24:48 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:24:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 245.6 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.2%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 56x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:24:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 245.1 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.6%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:25:01 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 244.9 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.9%, Prefix cache hit rate: 91.6% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:25:03 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:25:06 [loggers.py:259] Engine 000: Avg prompt throughput: 35.7 tokens/s, Avg generation throughput: 277.8 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.9%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:25:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 292.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.3%, Prefix cache hit rate: 91.6% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:25:16 [loggers.py:259] Engine 000: Avg prompt throughput: 113.7 tokens/s, Avg generation throughput: 373.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.8%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:25:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 389.0 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.4%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458337, ip=10.128.16.42) ERROR 06-06 18:25:21 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:25:24] OK: 351 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:25:24] OK: RSS 1.23 GiB | node mem 440.4/858.0 GiB used (51.3%), avail 417.6 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:25:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 369.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.1%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:25:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 343.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.6%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:25:32] OK: 359 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:25:32] OK: RSS 1.25 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) INFO 06-06 18:25:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.6 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.1%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:25:41] OK: 341 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:25:41] OK: RSS 1.28 GiB | node mem 396.8/858.0 GiB used (46.2%), avail 461.2 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429366, ip=10.128.16.47) INFO 06-06 18:25:41 [loggers.py:259] Engine 000: Avg prompt throughput: 1.6 tokens/s, Avg generation throughput: 416.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.8%, Prefix cache hit rate: 87.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) INFO 06-06 18:25:46 [loggers.py:259] Engine 000: Avg prompt throughput: 42.5 tokens/s, Avg generation throughput: 273.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.1%, Prefix cache hit rate: 90.1% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:25:49] OK: 387 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:25:49] OK: RSS 1.28 GiB | node mem 380.6/858.0 GiB used (44.4%), avail 477.4 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) INFO 06-06 18:25:51 [loggers.py:259] Engine 000: Avg prompt throughput: 9.6 tokens/s, Avg generation throughput: 317.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.6%, Prefix cache hit rate: 90.1% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:25:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 632.2 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 88.0%, Prefix cache hit rate: 89.3% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:26:00 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:01 [loggers.py:259] Engine 000: Avg prompt throughput: 144.9 tokens/s, Avg generation throughput: 651.0 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 87.9%, Prefix cache hit rate: 89.2% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:06 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:06 [loggers.py:259] Engine 000: Avg prompt throughput: 432.2 tokens/s, Avg generation throughput: 643.0 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 88.7%, Prefix cache hit rate: 88.8% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1437455, ip=10.128.16.36) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:26:08] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:26:08] OK: RSS 1.88 GiB | node mem 213.9/858.0 GiB used (24.9%), avail 644.1 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:11 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 666.3 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 89.7%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 661.6 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 90.6%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 656.4 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 91.5%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 652.4 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 92.4%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) INFO 06-06 18:26:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 488.8 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 59.4%, Prefix cache hit rate: 88.8% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1101390, ip=10.128.16.38) INFO 06-06 18:26:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 488.2 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.1%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:37 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:26:41 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 640.2 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 95.1%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:26:44 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:46 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 635.3 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 87.1%, Prefix cache hit rate: 88.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:45 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:51 [loggers.py:259] Engine 000: Avg prompt throughput: 217.2 tokens/s, Avg generation throughput: 640.0 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 88.3%, Prefix cache hit rate: 88.7% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:26:56 [loggers.py:259] Engine 000: Avg prompt throughput: 42.8 tokens/s, Avg generation throughput: 633.8 tokens/s, Running: 18 reqs, Waiting: 0 reqs, GPU KV cache usage: 89.9%, Prefix cache hit rate: 88.7% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:26:59 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:27:01 [loggers.py:259] Engine 000: Avg prompt throughput: 188.8 tokens/s, Avg generation throughput: 656.7 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 90.3%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:27:06 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 650.4 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 91.3%, Prefix cache hit rate: 88.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:27:11 [loggers.py:259] Engine 000: Avg prompt throughput: 2137.9 tokens/s, Avg generation throughput: 586.7 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 95.1%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:27:12 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) INFO 06-06 18:27:16 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 628.3 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 83.0%, Prefix cache hit rate: 86.8% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:27:21 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 492.0 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.1%, Prefix cache hit rate: 89.3% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:27:24 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:27:24] OK: 452 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:27:24] OK: RSS 1.24 GiB | node mem 440.4/858.0 GiB used (51.3%), avail 417.6 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:27:26 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 492.4 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.8%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:27:31 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 491.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.5%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:27:32] OK: 363 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:27:32] OK: RSS 1.26 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) ERROR 06-06 18:27:34 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:27:36 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 464.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.5%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:27:41] OK: 339 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:27:41] OK: RSS 1.28 GiB | node mem 396.8/858.0 GiB used (46.2%), avail 461.2 GiB (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:27:41 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 452.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.1%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:27:46 [loggers.py:259] Engine 000: Avg prompt throughput: 62.2 tokens/s, Avg generation throughput: 470.8 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.9%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) ERROR 06-06 18:27:47 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:27:48] OK: 362 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:27:48] OK: RSS 1.28 GiB | node mem 383.1/858.0 GiB used (44.7%), avail 474.9 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:27:51 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 477.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.4%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:27:56 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:27:56 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.7 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.1%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:28:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 406.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.7%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:04 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:28:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 406.0 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.2%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (skyrl_entrypoint pid=1452083) [fd-monitor] [18:28:08] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:28:08] OK: RSS 1.89 GiB | node mem 214.0/858.0 GiB used (24.9%), avail 644.0 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1246434, ip=10.128.16.45) ERROR 06-06 18:28:05 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:28:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 405.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.8%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:28:17 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 406.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.3%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:28:21 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:28:22 [loggers.py:259] Engine 000: Avg prompt throughput: 2.8 tokens/s, Avg generation throughput: 443.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.0%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1436902, ip=10.128.16.36) ERROR 06-06 18:28:24 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:28:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 448.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.6%, Prefix cache hit rate: 89.3% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 39x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388352, ip=10.128.16.44) INFO 06-06 18:28:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 448.9 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.2%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 41x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:28:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 455.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.9%, Prefix cache hit rate: 92.4% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:28:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 27x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:28:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 427.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.5%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1387457, ip=10.128.16.37) ERROR 06-06 18:28:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:28:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 410.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.1%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:28:52 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 410.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.7%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:28:54 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:28:57 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 410.0 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.2%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] Error in preprocessing prompt inputs [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] Traceback (most recent call last): [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] return await asyncio.gather( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] raise VLLMValidationError( [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) ERROR 06-06 18:28:57 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:29:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 408.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 30.8%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388353, ip=10.128.16.44) ERROR 06-06 18:29:05 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:29:07 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 408.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.4%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:29:12 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 408.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.0%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455594, ip=10.128.16.46) ERROR 06-06 18:29:13 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:29:17 [loggers.py:259] Engine 000: Avg prompt throughput: 2.8 tokens/s, Avg generation throughput: 436.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.8%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:15 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:29:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 453.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.4%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:29:19 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:29:24] OK: 351 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:29:24] OK: RSS 1.23 GiB | node mem 440.4/858.0 GiB used (51.3%), avail 417.6 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:29:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 453.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.0%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098012, ip=10.128.16.43) ERROR 06-06 18:29:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 49x across cluster] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:29:32] OK: 353 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:29:32] OK: RSS 1.27 GiB | node mem 439.1/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:29:32 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 453.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.7%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:31 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 56x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:29:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 453.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.3%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:29:41] OK: 333 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:29:41] OK: RSS 1.29 GiB | node mem 396.9/858.0 GiB used (46.3%), avail 461.1 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:29:42 [loggers.py:259] Engine 000: Avg prompt throughput: 42.9 tokens/s, Avg generation throughput: 443.9 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.0%, Prefix cache hit rate: 89.0% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] Error in preprocessing prompt inputs [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] Traceback (most recent call last): [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] return await asyncio.gather( [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] raise VLLMValidationError( [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1458336, ip=10.128.16.42) ERROR 06-06 18:29:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 8x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:29:47 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 479.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.6%, Prefix cache hit rate: 89.0% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:29:49] OK: 372 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:29:49] OK: RSS 1.29 GiB | node mem 383.1/858.0 GiB used (44.7%), avail 474.9 GiB (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:29:45 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:29:52 [loggers.py:259] Engine 000: Avg prompt throughput: 45.4 tokens/s, Avg generation throughput: 440.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.5%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:29:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:29:57 [loggers.py:259] Engine 000: Avg prompt throughput: 20.2 tokens/s, Avg generation throughput: 478.1 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.1%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:29:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:02 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 411.8 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.2%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) ERROR 06-06 18:29:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:07 [loggers.py:259] Engine 000: Avg prompt throughput: 83.4 tokens/s, Avg generation throughput: 391.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 46.8%, Prefix cache hit rate: 92.6% [repeated 48x across cluster] (skyrl_entrypoint pid=1452083) [fd-monitor] [18:30:08] OK: 565 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:30:08] OK: RSS 1.89 GiB | node mem 214.0/858.0 GiB used (24.9%), avail 644.0 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455593, ip=10.128.16.46) ERROR 06-06 18:30:06 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:30:11 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:12 [loggers.py:259] Engine 000: Avg prompt throughput: 35.3 tokens/s, Avg generation throughput: 469.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.3%, Prefix cache hit rate: 92.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:14 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 26x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:17 [loggers.py:259] Engine 000: Avg prompt throughput: 138.0 tokens/s, Avg generation throughput: 431.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.2%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098140, ip=10.128.16.43) INFO 06-06 18:30:22 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 501.4 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.4%, Prefix cache hit rate: 92.3% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:30:27 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 333.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.6%, Prefix cache hit rate: 89.2% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (AsyncVLLMInferenceEngine pid=1387587, ip=10.128.16.37) INFO 06-06 18:30:32 [loggers.py:259] Engine 000: Avg prompt throughput: 21.3 tokens/s, Avg generation throughput: 343.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 29.7%, Prefix cache hit rate: 89.3% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:37 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 460.5 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.3%, Prefix cache hit rate: 92.5% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:42 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 460.2 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.0%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:48 [loggers.py:259] Engine 000: Avg prompt throughput: 149.2 tokens/s, Avg generation throughput: 472.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.6%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:53 [loggers.py:259] Engine 000: Avg prompt throughput: 78.2 tokens/s, Avg generation throughput: 426.6 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.8%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:30:55 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:30:58 [loggers.py:259] Engine 000: Avg prompt throughput: 62.7 tokens/s, Avg generation throughput: 478.1 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.1%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 506.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.8%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 506.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.5%, Prefix cache hit rate: 92.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:11 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:13 [loggers.py:259] Engine 000: Avg prompt throughput: 25.5 tokens/s, Avg generation throughput: 524.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.4%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:18 [loggers.py:259] Engine 000: Avg prompt throughput: 95.5 tokens/s, Avg generation throughput: 505.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.2%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:31:20 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 504.7 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.9%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:31:24] OK: 363 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:31:24] OK: RSS 1.24 GiB | node mem 440.4/858.0 GiB used (51.3%), avail 417.6 GiB (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:31:20 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:28 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 468.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.6%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:31:32] OK: 358 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:31:32] OK: RSS 1.27 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 451.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.4%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) ERROR 06-06 18:31:33 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 60x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:38 [loggers.py:259] Engine 000: Avg prompt throughput: 11.4 tokens/s, Avg generation throughput: 394.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.1%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429139, ip=10.128.16.41) ERROR 06-06 18:31:38 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:31:41] OK: 323 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:31:41] OK: RSS 1.29 GiB | node mem 396.9/858.0 GiB used (46.3%), avail 461.1 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:43 [loggers.py:259] Engine 000: Avg prompt throughput: 92.3 tokens/s, Avg generation throughput: 429.4 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.7%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 461.8 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.3%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:31:49] OK: 371 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:31:49] OK: RSS 1.30 GiB | node mem 383.1/858.0 GiB used (44.7%), avail 474.8 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:53 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 461.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.0%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:31:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 461.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.6%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:32:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 439.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.0%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:32:08] OK: 565 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:32:08] OK: RSS 1.90 GiB | node mem 214.0/858.0 GiB used (24.9%), avail 644.0 GiB (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:32:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.4 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 34.6%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:32:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.0 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.2%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1098141, ip=10.128.16.43) INFO 06-06 18:32:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 35.7%, Prefix cache hit rate: 92.5% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1246562, ip=10.128.16.45) ERROR 06-06 18:32:23 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:32:23 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 414.8 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.9%, Prefix cache hit rate: 90.6% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 9x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:32:25 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:32:28 [loggers.py:259] Engine 000: Avg prompt throughput: 22.6 tokens/s, Avg generation throughput: 370.7 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.1%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:32:33 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 376.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.6%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:32:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.3%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:32:39 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:32:43 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 285.1 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.3%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:32:42 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:32:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 283.4 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.7%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:32:50 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:32:53 [loggers.py:259] Engine 000: Avg prompt throughput: 88.3 tokens/s, Avg generation throughput: 346.0 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.4%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:32:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 340.9 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.2%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:32:59 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 317.5 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.4%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:08 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 283.4 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 19.8%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 283.7 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.2%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 283.6 tokens/s, Running: 6 reqs, Waiting: 0 reqs, GPU KV cache usage: 20.6%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 30x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:23 [loggers.py:259] Engine 000: Avg prompt throughput: 8.2 tokens/s, Avg generation throughput: 284.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.1%, Prefix cache hit rate: 90.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:33:24] OK: 419 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:33:24] OK: RSS 1.25 GiB | node mem 440.4/858.0 GiB used (51.3%), avail 417.5 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:33:27 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:28 [loggers.py:259] Engine 000: Avg prompt throughput: 82.0 tokens/s, Avg generation throughput: 366.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.5%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:33:32] OK: 357 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:33:32] OK: RSS 1.27 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 419.0 GiB (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:33 [loggers.py:259] Engine 000: Avg prompt throughput: 9.4 tokens/s, Avg generation throughput: 363.1 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.2%, Prefix cache hit rate: 90.7% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 343.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.5%, Prefix cache hit rate: 90.7% [repeated 49x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:33:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:33:41] OK: 336 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:33:41] OK: RSS 1.29 GiB | node mem 396.9/858.0 GiB used (46.3%), avail 461.1 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1455465, ip=10.128.16.46) INFO 06-06 18:33:43 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 331.0 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 22.9%, Prefix cache hit rate: 90.7% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:33:48 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 345.2 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.8%, Prefix cache hit rate: 91.4% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:33:49] OK: 361 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:33:49] OK: RSS 1.30 GiB | node mem 383.1/858.0 GiB used (44.7%), avail 474.8 GiB (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437031, ip=10.128.16.36) ERROR 06-06 18:33:49 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 18x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:33:53 [loggers.py:259] Engine 000: Avg prompt throughput: 47.6 tokens/s, Avg generation throughput: 333.0 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.5%, Prefix cache hit rate: 91.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:33:58 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 361.8 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.0%, Prefix cache hit rate: 91.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:34:03 [loggers.py:259] Engine 000: Avg prompt throughput: 0.2 tokens/s, Avg generation throughput: 381.9 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.5%, Prefix cache hit rate: 91.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 7x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:34:07 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:34:08] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:34:08] OK: RSS 1.90 GiB | node mem 213.9/858.0 GiB used (24.9%), avail 644.0 GiB (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:34:08 [loggers.py:259] Engine 000: Avg prompt throughput: 9.4 tokens/s, Avg generation throughput: 441.1 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.3%, Prefix cache hit rate: 91.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:34:13 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 448.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 51.9%, Prefix cache hit rate: 91.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1101519, ip=10.128.16.38) INFO 06-06 18:34:18 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 449.9 tokens/s, Running: 10 reqs, Waiting: 0 reqs, GPU KV cache usage: 52.6%, Prefix cache hit rate: 91.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 11x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) INFO 06-06 18:34:24 [loggers.py:259] Engine 000: Avg prompt throughput: 23.1 tokens/s, Avg generation throughput: 573.1 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 53.9%, Prefix cache hit rate: 92.2% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:34:26 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) INFO 06-06 18:34:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 563.8 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 54.2%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 28x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) INFO 06-06 18:34:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 551.3 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.0%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 52x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) INFO 06-06 18:34:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 551.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 55.7%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 31x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1388222, ip=10.128.16.44) ERROR 06-06 18:34:40 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1953614, ip=10.128.16.187) INFO 06-06 18:34:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 550.7 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 56.5%, Prefix cache hit rate: 92.2% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 6x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:34:47 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:34:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 694.0 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.4%, Prefix cache hit rate: 91.0% [repeated 49x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:34:54 [loggers.py:259] Engine 000: Avg prompt throughput: 107.2 tokens/s, Avg generation throughput: 643.2 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 61.9%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:34:59 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 631.9 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 62.8%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:04 [loggers.py:259] Engine 000: Avg prompt throughput: 35.9 tokens/s, Avg generation throughput: 633.9 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 63.3%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:09 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 634.8 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.2%, Prefix cache hit rate: 90.9% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 676.4 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.3%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 660.5 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.5%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 581.1 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 64.8%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:35:24] OK: 381 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:35:24] OK: RSS 1.25 GiB | node mem 440.4/858.0 GiB used (51.3%), avail 417.5 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 564.7 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.6%, Prefix cache hit rate: 90.9% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:35:32] OK: 354 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:35:32] OK: RSS 1.27 GiB | node mem 438.9/858.0 GiB used (51.2%), avail 419.0 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:34 [loggers.py:259] Engine 000: Avg prompt throughput: 1.4 tokens/s, Avg generation throughput: 598.2 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 66.4%, Prefix cache hit rate: 91.0% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 610.0 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 67.3%, Prefix cache hit rate: 91.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:35:41] OK: 315 / 131,072 FDs open (0.2% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:35:41] OK: RSS 1.30 GiB | node mem 396.9/858.0 GiB used (46.3%), avail 461.0 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 609.7 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 68.1%, Prefix cache hit rate: 91.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:35:49] OK: 360 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:35:49] OK: RSS 1.30 GiB | node mem 383.2/858.0 GiB used (44.7%), avail 474.8 GiB (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:49 [loggers.py:259] Engine 000: Avg prompt throughput: 102.1 tokens/s, Avg generation throughput: 656.1 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 70.0%, Prefix cache hit rate: 91.0% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:54 [loggers.py:259] Engine 000: Avg prompt throughput: 132.5 tokens/s, Avg generation throughput: 655.4 tokens/s, Running: 18 reqs, Waiting: 0 reqs, GPU KV cache usage: 72.2%, Prefix cache hit rate: 91.0% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:35:59 [loggers.py:259] Engine 000: Avg prompt throughput: 134.7 tokens/s, Avg generation throughput: 728.3 tokens/s, Running: 19 reqs, Waiting: 0 reqs, GPU KV cache usage: 72.9%, Prefix cache hit rate: 91.0% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 5x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1437030, ip=10.128.16.36) ERROR 06-06 18:36:03 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:36:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 667.5 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 65.8%, Prefix cache hit rate: 91.0% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:36:08] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:36:08] OK: RSS 1.90 GiB | node mem 213.9/858.0 GiB used (24.9%), avail 644.1 GiB (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:36:09 [loggers.py:259] Engine 000: Avg prompt throughput: 69.5 tokens/s, Avg generation throughput: 634.7 tokens/s, Running: 17 reqs, Waiting: 0 reqs, GPU KV cache usage: 66.1%, Prefix cache hit rate: 91.0% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) INFO 06-06 18:36:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 360.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.0%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) INFO 06-06 18:36:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 360.2 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.5%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) INFO 06-06 18:36:24 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 358.8 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.0%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 6x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) INFO 06-06 18:36:29 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 359.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.5%, Prefix cache hit rate: 90.4% [repeated 47x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 22x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1437029, ip=10.128.16.36) INFO 06-06 18:36:34 [loggers.py:259] Engine 000: Avg prompt throughput: 152.3 tokens/s, Avg generation throughput: 372.2 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 50.8%, Prefix cache hit rate: 90.3% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 36x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) ERROR 06-06 18:36:39 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:36:39 [loggers.py:259] Engine 000: Avg prompt throughput: 58.0 tokens/s, Avg generation throughput: 380.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.0%, Prefix cache hit rate: 91.5% [repeated 50x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 5x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] Error in preprocessing prompt inputs [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] Traceback (most recent call last): [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 4x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] return await asyncio.gather( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] tokens = validator(tokenizer, tokens) [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] raise VLLMValidationError( [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429270, ip=10.128.16.41) ERROR 06-06 18:36:39 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:36:44 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 400.6 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 31.6%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:36:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 400.5 tokens/s, Running: 9 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.1%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:36:54 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 369.6 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.0%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:36:59 [loggers.py:259] Engine 000: Avg prompt throughput: 67.6 tokens/s, Avg generation throughput: 353.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 33.8%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 21x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:37:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 333.3 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 32.6%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) INFO 06-06 18:37:09 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 688.0 tokens/s, Running: 18 reqs, Waiting: 0 reqs, GPU KV cache usage: 67.6%, Prefix cache hit rate: 89.0% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 24x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458338, ip=10.128.16.42) ERROR 06-06 18:37:11 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) INFO 06-06 18:37:14 [loggers.py:259] Engine 000: Avg prompt throughput: 194.5 tokens/s, Avg generation throughput: 663.9 tokens/s, Running: 18 reqs, Waiting: 0 reqs, GPU KV cache usage: 69.7%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=1101517, ip=10.128.16.38) INFO 06-06 18:37:19 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 664.0 tokens/s, Running: 18 reqs, Waiting: 0 reqs, GPU KV cache usage: 70.7%, Prefix cache hit rate: 89.1% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1098142, ip=10.128.16.43) ERROR 06-06 18:37:23 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:37:24] OK: 353 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1437455, ip=10.128.16.36) [fd-monitor] [18:37:24] OK: RSS 1.26 GiB | node mem 440.4/858.0 GiB used (51.3%), avail 417.5 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:37:24 [loggers.py:259] Engine 000: Avg prompt throughput: 154.4 tokens/s, Avg generation throughput: 455.3 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 36.8%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429368, ip=10.128.16.47) ERROR 06-06 18:37:26 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 3x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 3x across cluster] (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:37:29 [loggers.py:259] Engine 000: Avg prompt throughput: 184.0 tokens/s, Avg generation throughput: 505.9 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.5%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 15x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:37:32] OK: 361 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1102057, ip=10.128.16.38) [fd-monitor] [18:37:32] OK: RSS 1.28 GiB | node mem 439.0/858.0 GiB used (51.2%), avail 418.9 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:37:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.4 tokens/s, Avg generation throughput: 522.0 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.2%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1458206, ip=10.128.16.42) ERROR 06-06 18:37:37 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:37:39 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 507.6 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.5%, Prefix cache hit rate: 91.6% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 17x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:37:41] OK: 343 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1429749, ip=10.128.16.41) [fd-monitor] [18:37:41] OK: RSS 1.30 GiB | node mem 397.0/858.0 GiB used (46.3%), avail 461.0 GiB (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:37:44 [loggers.py:259] Engine 000: Avg prompt throughput: 21.5 tokens/s, Avg generation throughput: 639.9 tokens/s, Running: 16 reqs, Waiting: 0 reqs, GPU KV cache usage: 49.5%, Prefix cache hit rate: 90.9% [repeated 47x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 32x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1953616, ip=10.128.16.187) ERROR 06-06 18:37:48 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:37:49] OK: 376 / 131,072 FDs open (0.3% of soft limit, hard limit: 131,072) (RolloutCoordinator pid=1388832, ip=10.128.16.44) [fd-monitor] [18:37:49] OK: RSS 1.30 GiB | node mem 384.4/858.0 GiB used (44.8%), avail 473.5 GiB (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:37:49 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 491.3 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 39.2%, Prefix cache hit rate: 91.6% [repeated 49x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 4x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=1429269, ip=10.128.16.41) ERROR 06-06 18:37:54 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:37:54 [loggers.py:259] Engine 000: Avg prompt throughput: 212.5 tokens/s, Avg generation throughput: 471.6 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 40.9%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. [repeated 2x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:37:59 [loggers.py:259] Engine 000: Avg prompt throughput: 33.6 tokens/s, Avg generation throughput: 591.3 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 41.9%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 8x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:38:04 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 616.3 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 42.3%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:38:08] OK: 564 / 131,072 FDs open (0.4% of soft limit, hard limit: 131,072) (skyrl_entrypoint pid=1452083) [fd-monitor] [18:38:08] OK: RSS 1.91 GiB | node mem 214.0/858.0 GiB used (24.9%), avail 644.0 GiB (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:38:09 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 576.3 tokens/s, Running: 13 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.1%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 10x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131332, ip=10.128.16.185) ERROR 06-06 18:38:13 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:38:14 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 561.7 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 43.2%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:18 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:38:19 [loggers.py:259] Engine 000: Avg prompt throughput: 225.5 tokens/s, Avg generation throughput: 504.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.5%, Prefix cache hit rate: 91.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 35x across cluster] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] Error in preprocessing prompt inputs (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] Traceback (most recent call last): (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 299, in render_chat_request (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] conversation, engine_prompts = await self._preprocess_chat( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 1018, in _preprocess_chat (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] (conversation,), (engine_prompt,) = await renderer.render_chat_async( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 381, in render_chat_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] tok_prompts = await self.tokenize_prompts_async(dict_prompts, tok_params) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 278, in tokenize_prompts_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] return await asyncio.gather( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/protocol.py", line 271, in tokenize_prompt_async (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] return params.apply_post_tokenization(self.tokenizer, prompt) # type: ignore[arg-type] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 373, in apply_post_tokenization (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] prompt["prompt_token_ids"] = self._validate_tokens( # type: ignore[typeddict-unknown-key] (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 357, in _validate_tokens (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] tokens = validator(tokenizer, tokens) (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/vllm/renderers/params.py", line 337, in _token_len_check (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] raise VLLMValidationError( (AsyncVLLMInferenceEngine pid=2131461, ip=10.128.16.185) ERROR 06-06 18:38:20 [serving.py:315] vllm.exceptions.VLLMValidationError: You passed 32769 input tokens and requested 0 output tokens. However, the model's context length is only 32768 tokens, resulting in a maximum input length of 32768 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=32769) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:38:24 [loggers.py:259] Engine 000: Avg prompt throughput: 10.4 tokens/s, Avg generation throughput: 535.8 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 44.2%, Prefix cache hit rate: 91.5% [repeated 49x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 23x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:38:30 [loggers.py:259] Engine 000: Avg prompt throughput: 54.9 tokens/s, Avg generation throughput: 579.8 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.5%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:38:35 [loggers.py:259] Engine 000: Avg prompt throughput: 36.2 tokens/s, Avg generation throughput: 647.5 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 47.4%, Prefix cache hit rate: 91.5% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 12x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1388832, ip=10.128.16.44) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (AsyncVLLMInferenceEngine pid=2131462, ip=10.128.16.185) INFO 06-06 18:38:40 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 665.7 tokens/s, Running: 15 reqs, Waiting: 0 reqs, GPU KV cache usage: 48.4%, Prefix cache hit rate: 91.5% [repeated 47x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:38:45 [loggers.py:259] Engine 000: Avg prompt throughput: 171.4 tokens/s, Avg generation throughput: 305.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.3%, Prefix cache hit rate: 90.5% [repeated 50x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 14x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:38:50 [loggers.py:259] Engine 000: Avg prompt throughput: 154.1 tokens/s, Avg generation throughput: 321.3 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.0%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) Provider List: https://docs.litellm.ai/docs/providers [repeated 16x across cluster] (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1429749, ip=10.128.16.41) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:38:55 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 355.4 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.5%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) Provider List: https://docs.litellm.ai/docs/providers [repeated 13x across cluster] (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:39:00 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 329.4 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 27.6%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Provider List: https://docs.litellm.ai/docs/providers [repeated 20x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1102057, ip=10.128.16.38) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1388832, ip=10.128.16.44) (RolloutCoordinator pid=1102057, ip=10.128.16.38) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1102057, ip=10.128.16.38) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=2131460, ip=10.128.16.185) INFO 06-06 18:39:05 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 312.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 28.0%, Prefix cache hit rate: 90.4% [repeated 48x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 39x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new (RolloutCoordinator pid=1437455, ip=10.128.16.36) LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1429749, ip=10.128.16.41) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (RolloutCoordinator pid=1437455, ip=10.128.16.36) (AsyncVLLMInferenceEngine pid=1246693, ip=10.128.16.45) INFO 06-06 18:39:09 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 672.7 tokens/s, Running: 14 reqs, Waiting: 0 reqs, GPU KV cache usage: 58.4%, Prefix cache hit rate: 90.9% [repeated 45x across cluster] (RolloutCoordinator pid=1437455, ip=10.128.16.36) Provider List: https://docs.litellm.ai/docs/providers [repeated 33x across cluster] Error executing job with overrides: ['+terminal_bench_config=terminal_bench', 'trainer.strategy=fsdp2', 'trainer.algorithm.advantage_estimator=rloo_n', 'trainer.algorithm.use_kl_loss=false', 'trainer.algorithm.kl_loss_coef=0.0', 'trainer.algorithm.eps_clip_low=0.2', 'trainer.algorithm.eps_clip_high=0.05', 'trainer.algorithm.loss_reduction=seq_mean_token_sum_norm_global', 'trainer.epochs=2', 'trainer.max_steps=80', 'trainer.update_epochs_per_batch=1', 'trainer.train_batch_size=64', 'trainer.policy_mini_batch_size=64', 'trainer.eval_batch_size=64', 'trainer.micro_forward_batch_size_per_gpu=4', 'trainer.micro_train_batch_size_per_gpu=1', 'trainer.max_prompt_length=999999', 'trainer.eval_interval=999999', 'trainer.eval_before_train=false', 'trainer.ckpt_interval=2', 'trainer.resume_mode=latest', 'trainer.hf_save_interval=5', '++trainer.hf_hub_repo_id=laion/ablation-pymethods2test-seqnorm', '++trainer.hf_hub_private=false', '++trainer.hf_hub_revision=main', '++trainer.enable_db_registration=false', 'trainer.project_name=OpenThoughts-Agent', 'trainer.log_level=INFO', 'trainer.tracker_commit_each_step=true', 'trainer.logger=console', 'trainer.run_name=ablation-pymethods2test-seqnorm', 'trainer.ckpt_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/checkpoints', 'trainer.export_path=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/exports', 'trainer.policy.optimizer_config.lr=8e-6', 'trainer.policy.optimizer_config.weight_decay=0.0', 'trainer.policy.optimizer_config.adam_betas=[0.9,0.999]', 'trainer.policy.optimizer_config.max_grad_norm=0.9', 'trainer.policy.fsdp_config.cpu_offload=false', 'trainer.policy.fsdp_config.reshard_after_forward=true', 'trainer.policy.fsdp_config.fsdp_size=4', 'trainer.policy.model.path=/e/home/jusers/feuer1/jupiter/.cache/huggingface/hub/models--laion--GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink/snapshots/0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', 'trainer.ref.fsdp_config.cpu_offload=false', 'trainer.ref.fsdp_config.reshard_after_forward=true', 'trainer.ref.fsdp_config.fsdp_size=4', 'trainer.placement.colocate_all=false', 'trainer.placement.policy_num_nodes=2', 'trainer.placement.ref_num_nodes=2', 'trainer.placement.policy_num_gpus_per_node=4', 'trainer.placement.ref_num_gpus_per_node=4', 'trainer.fully_async.max_staleness_steps=16', 'trainer.fully_async.num_parallel_generation_workers=338', 'generator.backend=vllm', 'generator.timeout_multiplier=1.0', 'generator.model_dtype=bfloat16', 'generator.inference_engine_tensor_parallel_size=1', 'generator.num_inference_engines=48', 'generator.n_samples_per_prompt=8', 'generator.eval_n_samples_per_prompt=8', 'generator.gpu_memory_utilization=0.75', 'generator.max_num_seqs=24', 'generator.max_num_batched_tokens=65536', 'generator.enable_prefix_caching=true', 'generator.enable_chunked_prefill=true', 'generator.run_engines_locally=true', 'generator.weight_sync_backend=nccl', 'generator.async_engine=true', 'generator.batched=false', 'generator.enable_http_endpoint=true', 'generator.enable_ray_prometheus_stats=false', 'generator.vllm_stats_interval=1', 'generator.append_eos_token_after_stop_str_in_multi_turn=true', 'generator.max_turns=999999', 'generator.sampling_params.max_generate_length=4096', 'generator.sampling_params.temperature=0.7', 'generator.sampling_params.top_p=0.95', 'generator.sampling_params.top_k=20', '++generator.engine_init_kwargs.max_model_len=32768', '++generator.engine_init_kwargs.custom_chat_template_chat_completion_path=chat_templates/qwen3_thinking_acc.jinja2', '++generator.engine_init_kwargs.served_model_name=0e3bff0c4e51f6b9ec0713b98b9eec36efb91cc6', 'data.train_data=["/e/scratch/jureap59/feuer1/tasks/exp_rpt_pymethods2test-large"]', 'data.val_data=[]', '+terminal_bench_config.trials_dir=/e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/trace_jobs', '+terminal_bench_config.harbor.name=terminus-2', '+terminal_bench_config.harbor.max_episodes=999999', '+terminal_bench_config.harbor.enable_summarize=false', '+terminal_bench_config.harbor.store_all_messages=true', '+terminal_bench_config.harbor.trajectory_config.raw_content=true', '+terminal_bench_config.harbor.enable_episode_logging=false', '+terminal_bench_config.harbor.record_terminal_session=false', '+terminal_bench_config.harbor.enable_pane_logging=false', '+terminal_bench_config.harbor.strict_json_parser=true', '+terminal_bench_config.harbor.interleaved_thinking=true', '+terminal_bench_config.harbor.extra_body.chat_template_kwargs.enable_thinking=true', '+terminal_bench_config.harbor.override_timeout_sec=900', '+terminal_bench_config.harbor.override_cpus=1', '+terminal_bench_config.harbor.override_memory_mb=2048', '+terminal_bench_config.harbor.override_storage_mb=2048', '+terminal_bench_config.harbor.auto_snapshot=true', '+terminal_bench_config.harbor.verifier_override_timeout_sec=120', '+terminal_bench_config.harbor.max_retries=3', '+terminal_bench_config.harbor.min_wait_sec=60.0', '+terminal_bench_config.harbor.max_wait_sec=600.0', '+terminal_bench_config.harbor.wait_multiplier=2.0', '+terminal_bench_config.harbor.exclude_exceptions=["VerifierTimeoutError","VerifierRuntimeError","RewardFileNotFoundError","RewardFileEmptyError","VerifierOutputParseError"]', '+terminal_bench_config.harbor.n_concurrent_trials=675', '+terminal_bench_config.harbor.log_level=INFO', '+terminal_bench_config.harbor.enable_reward_shaping=false', '+terminal_bench_config.harbor.enable_error_classification=true', '+terminal_bench_config.harbor.mask_exceptions=["DaytonaError","EnvironmentStartTimeoutError","NetworkError","ConnectionError","RewardFileNotFoundError","RewardFileEmptyError","AgentEnvironmentTimeoutError","ContextLengthExceededError"]', '+terminal_bench_config.harbor.default_error_treatment=zero', '+terminal_bench_config.harbor.passthrough_exceptions=["AgentTimeoutError"]', '+terminal_bench_config.harbor.zero_exceptions=[]', '+terminal_bench_config.model_info.max_input_tokens=32000', '+terminal_bench_config.model_info.max_output_tokens=4096', '+terminal_bench_config.archiving.enabled=false', '+terminal_bench_config.trace_upload.enabled=true', '+terminal_bench_config.trace_upload.repo_org=DCAgent', '+terminal_bench_config.trace_upload.episodes=last', '+terminal_bench_config.trace_upload.dataset_type=SFT', '+terminal_bench_config.trace_upload.cleanup=true'] Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/entrypoints/main_tbench.py", line 142, in main() File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/main.py", line 94, in decorated_main _run_hydra( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra _run_app( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 457, in _run_app run_and_report( File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 223, in run_and_report raise ex File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 220, in run_and_report return func() ^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/utils.py", line 458, in lambda: hydra.run( ^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/_internal/hydra.py", line 132, in run _ = ret.return_value ^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/core/utils.py", line 260, in return_value raise self._return_value File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/hydra/core/utils.py", line 186, in run_job ret.return_value = task_function(task_cfg) ^^^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/SkyRL/skyrl-train/examples/terminal_bench/entrypoints/main_tbench.py", line 132, in main ray.get(skyrl_entrypoint.remote(cfg)) File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py", line 2967, in get values, debugger_breakpoint = worker.get_objects( ^^^^^^^^^^^^^^^^^^^ File "/e/scratch/jureap59/feuer1/OpenThoughts-Agent/envs/rl/lib/python3.12/site-packages/ray/_private/worker.py", line 1017, in get_objects raise value ray.exceptions.NodeDiedError: Task failed because the node it was running on is dead or unavailable. Node IP: 10.128.16.35, node ID: 83bfd4a4d537552ce09528616509a646e00fecc0b97c317d9165332d. This can happen if the node was preempted, had a hardware failure, or its raylet crashed unexpectedly. To see node death information, use `ray list nodes --filter node_id=83bfd4a4d537552ce09528616509a646e00fecc0b97c317d9165332d`, check the Ray dashboard cluster page, search the node ID in the GCS logs, or use `ray logs raylet.out -ip 10.128.16.35`. Collecting Ray logs from worker jpbo-001-36... WARNING: Failed to collect logs from jpbo-001-36 Collecting Ray logs from worker jpbo-001-37... WARNING: Failed to collect logs from jpbo-001-37 Collecting Ray logs from worker jpbo-001-38... Stopping Ray cluster... WARNING: Failed to collect logs from jpbo-001-38 Collecting Ray logs from worker jpbo-001-41... WARNING: Failed to collect logs from jpbo-001-41 Collecting Ray logs from worker jpbo-001-42... WARNING: Failed to collect logs from jpbo-001-42 Collecting Ray logs from worker jpbo-001-43... Ray cluster stopped [RLJobRunner] Crash detected (exit!=0) — preserving Ray logs to /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm/ray_logs BEFORE trace upload (so a wall-clock kill can't lose crash evidence)... WARNING: Failed to collect logs from jpbo-001-43 Collecting Ray logs from worker jpbo-001-44... Collecting Ray logs from worker jpbo-001-36... WARNING: Failed to collect logs from jpbo-001-44 Collecting Ray logs from worker jpbo-001-45... WARNING: Failed to collect logs from jpbo-001-36 Collecting Ray logs from worker jpbo-001-37... WARNING: Failed to collect logs from jpbo-001-37 Collecting Ray logs from worker jpbo-001-38... WARNING: Failed to collect logs from jpbo-001-38 Collecting Ray logs from worker jpbo-001-41... WARNING: Failed to collect logs from jpbo-001-45 Collecting Ray logs from worker jpbo-001-46... WARNING: Failed to collect logs from jpbo-001-41 Collecting Ray logs from worker jpbo-001-42... WARNING: Failed to collect logs from jpbo-001-42 Collecting Ray logs from worker jpbo-001-43... WARNING: Failed to collect logs from jpbo-001-43 Collecting Ray logs from worker jpbo-001-44... WARNING: Failed to collect logs from jpbo-001-46 Collecting Ray logs from worker jpbo-001-47... WARNING: Failed to collect logs from jpbo-001-44 Collecting Ray logs from worker jpbo-001-45... WARNING: Failed to collect logs from jpbo-001-45 Collecting Ray logs from worker jpbo-001-46... WARNING: Failed to collect logs from jpbo-001-46 Collecting Ray logs from worker jpbo-001-47... WARNING: Failed to collect logs from jpbo-001-47 Collecting Ray logs from worker jpbo-001-48... WARNING: Failed to collect logs from jpbo-001-47 Collecting Ray logs from worker jpbo-001-48... WARNING: Failed to collect logs from jpbo-001-48 Collecting Ray logs from worker jpbo-004-41... WARNING: Failed to collect logs from jpbo-001-48 Collecting Ray logs from worker jpbo-004-41... WARNING: Failed to collect logs from jpbo-004-41 Collecting Ray logs from worker jpbo-004-43... WARNING: Failed to collect logs from jpbo-004-41 Collecting Ray logs from worker jpbo-004-43... WARNING: Failed to collect logs from jpbo-004-43 WARNING: Failed to collect logs from jpbo-004-43 Ray log preservation complete [RLJobRunner] Crash-time Ray log preservation complete. [RLJobRunner] Launching trace upload (training exit code: 1): repo_id: DCAgent/ablation-pymethods2test-seqnorm job_dir: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/ablation-pymethods2test-seqnorm episodes: last log: /e/data1/datasets/playground/ot-baf/ablation-pymethods2test-seqnorm_2/logs/ablation-pymethods2test-seqnorm_trace_upload.log [RLJobRunner] Waiting for trace upload to complete...