diff --git a/dockerrizhi.txt b/dockerrizhi.txt
new file mode 100644
index 00000000..489ffabc
--- /dev/null
+++ b/dockerrizhi.txt
@@ -0,0 +1,3787 @@
+我们的最开始（只截取了部分，错误太多了）日志：
+2026-08-07T08:26:45.627314093Z /usr/local/corex/lib/python3/dist-packages/torch/cuda/__init__.py:51: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+2026-08-07T08:26:45.627350664Z   import pynvml  # type: ignore[import]
+2026-08-07T08:26:47.166771940Z INFO 08-07 08:26:47 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
+2026-08-07T08:26:48.657239181Z 2026-08-07 08:26:48.657168: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
+2026-08-07T08:26:48.710793117Z 2026-08-07 08:26:48.710745: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
+2026-08-07T08:26:48.710798808Z To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F AVX512_VNNI AVX512_BF16 AVX_VNNI AMX_TILE AMX_INT8 AMX_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
+2026-08-07T08:26:48.741894791Z WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
+2026-08-07T08:26:54.117398862Z INFO 08-07 08:26:54 api_server.py:530] vLLM API server version 0.6.3
+2026-08-07T08:26:54.117663353Z INFO 08-07 08:26:54 api_server.py:531] args: Namespace(host=None, port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=True, enable_auto_tool_choice=True, tool_call_parser='qwen3_coder', tool_parser_plugin='', reasoning_parser='qwen3', model='/model', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=True, download_dir=None, load_format='auto', config_format='auto', dtype='half', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=100000, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=4, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=True, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=0, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.9, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=1, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=True, max_context_len_to_capture=None, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['llm'], qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, override_neuron_config=None, scheduling_policy='fcfs', disable_log_requests=True, max_log_len=None, disable_fastapi_docs=False)
+2026-08-07T08:26:54.134049573Z INFO 08-07 08:26:54 config.py:1670] Downcasting torch.float32 to torch.float16.
+2026-08-07T08:27:04.772029418Z INFO 08-07 08:27:04 config.py:887] Defaulting to use mp for distributed inference
+2026-08-07T08:27:04.772339448Z WARNING 08-07 08:27:04 arg_utils.py:963] The model has a long context length (100000). This may cause OOM errors during the initial memory profiling phase, or result in low performance due to small KV cache space. Consider setting --max-model-len to a smaller value.
+2026-08-07T08:27:04.772466699Z WARNING 08-07 08:27:04 config.py:380] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
+2026-08-07T08:27:04.775198251Z INFO 08-07 08:27:04 llm_engine.py:237] Initializing an LLM engine (v0.6.3) with config: model='/model', speculative_config=None, tokenizer='/model', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=100000, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=4, pipeline_parallel_size=1, disable_custom_all_reduce=True, quantization=None, enforce_eager=True, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=llm, use_v2_block_manager=True, num_scheduler_steps=1, chunked_prefill_enabled=False multi_step_stream_outputs=True, enable_prefix_caching=True, use_async_output_proc=False, use_cached_outputs=False, mm_processor_kwargs=None)
+2026-08-07T08:27:05.270845986Z WARNING 08-07 08:27:05 multiproc_gpu_executor.py:53] Reducing Torch parallelism from 64 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
+2026-08-07T08:27:05.311613008Z INFO 08-07 08:27:05 custom_cache_manager.py:17] Setting Triton cache manager to: vllm.triton_utils.custom_cache_manager:CustomCacheManager
+2026-08-07T08:27:05.362472560Z INFO 08-07 08:27:05 selector.py:115] Using XFormers backend.
+2026-08-07T08:27:05.804983102Z /usr/local/corex/lib/python3/dist-packages/torch/cuda/__init__.py:51: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+2026-08-07T08:27:05.804993579Z   import pynvml  # type: ignore[import]
+2026-08-07T08:27:05.821100558Z /usr/local/corex/lib/python3/dist-packages/torch/cuda/__init__.py:51: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+2026-08-07T08:27:05.821117178Z   import pynvml  # type: ignore[import]
+2026-08-07T08:27:05.821119437Z /usr/local/corex/lib/python3/dist-packages/torch/cuda/__init__.py:51: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+2026-08-07T08:27:05.821121878Z   import pynvml  # type: ignore[import]
+2026-08-07T08:27:07.336019924Z INFO 08-07 08:27:07 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
+2026-08-07T08:27:07.343924685Z INFO 08-07 08:27:07 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
+2026-08-07T08:27:07.349602702Z INFO 08-07 08:27:07 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
+2026-08-07T08:27:08.923183763Z WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
+2026-08-07T08:27:08.923188692Z WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
+2026-08-07T08:27:08.923204313Z WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
+2026-08-07T08:27:14.476866220Z (VllmWorkerProcess pid=345) INFO 08-07 08:27:14 selector.py:115] Using XFormers backend.
+2026-08-07T08:27:14.481729878Z (VllmWorkerProcess pid=346) INFO 08-07 08:27:14 selector.py:115] Using XFormers backend.
+2026-08-07T08:27:14.483088386Z (VllmWorkerProcess pid=344) INFO 08-07 08:27:14 selector.py:115] Using XFormers backend.
+2026-08-07T08:27:14.497162083Z (VllmWorkerProcess pid=345) INFO 08-07 08:27:14 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
+2026-08-07T08:27:14.501909522Z (VllmWorkerProcess pid=346) INFO 08-07 08:27:14 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
+2026-08-07T08:27:14.503443561Z (VllmWorkerProcess pid=344) INFO 08-07 08:27:14 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
+2026-08-07T08:27:14.829071062Z INFO 08-07 08:27:14 shm_broadcast.py:242] vLLM message queue communication handle: Handle(connect_ip='127.0.0.1', local_reader_ranks=[1, 2, 3], buffer=<vllm.distributed.device_communicators.shm_broadcast.ShmRingBuffer object at 0x7f18a268d690>, local_subscribe_port=44159, remote_subscribe_port=None)
+2026-08-07T08:27:14.863285326Z INFO 08-07 08:27:14 model_runner.py:1119] Starting to load model /model...
+2026-08-07T08:27:14.863469254Z (VllmWorkerProcess pid=345) INFO 08-07 08:27:14 model_runner.py:1119] Starting to load model /model...
+2026-08-07T08:27:14.863555057Z (VllmWorkerProcess pid=344) INFO 08-07 08:27:14 model_runner.py:1119] Starting to load model /model...
+2026-08-07T08:27:14.863630707Z (VllmWorkerProcess pid=346) INFO 08-07 08:27:14 model_runner.py:1119] Starting to load model /model...
+2026-08-07T08:27:14.905405480Z INFO 08-07 08:27:14 selector.py:115] Using XFormers backend.
+2026-08-07T08:27:14.909672212Z (VllmWorkerProcess pid=344) INFO 08-07 08:27:14 selector.py:115] Using XFormers backend.
+2026-08-07T08:27:14.909769804Z (VllmWorkerProcess pid=345) INFO 08-07 08:27:14 selector.py:115] Using XFormers backend.
+2026-08-07T08:27:14.909838151Z (VllmWorkerProcess pid=346) INFO 08-07 08:27:14 selector.py:115] Using XFormers backend.
+2026-08-07T08:27:14.997480301Z 
+Loading safetensors checkpoint shards:   0% Completed | 0/16 [00:00<?, ?it/s]
+2026-08-07T08:27:16.312189167Z 
+Loading safetensors checkpoint shards:   6% Completed | 1/16 [00:01<00:19,  1.31s/it]
+2026-08-07T08:27:17.414772678Z 
+Loading safetensors checkpoint shards:  12% Completed | 2/16 [00:02<00:16,  1.19s/it]
+2026-08-07T08:27:17.943863295Z 
+Loading safetensors checkpoint shards:  19% Completed | 3/16 [00:02<00:11,  1.13it/s]
+2026-08-07T08:27:19.184723162Z 
+Loading safetensors checkpoint shards:  25% Completed | 4/16 [00:04<00:12,  1.03s/it]
+2026-08-07T08:27:20.167398294Z 
+Loading safetensors checkpoint shards:  31% Completed | 5/16 [00:05<00:11,  1.01s/it]
+2026-08-07T08:27:21.228769907Z 
+Loading safetensors checkpoint shards:  38% Completed | 6/16 [00:06<00:10,  1.03s/it]
+2026-08-07T08:27:22.321152523Z 
+Loading safetensors checkpoint shards:  44% Completed | 7/16 [00:07<00:09,  1.05s/it]
+2026-08-07T08:27:23.505443055Z 
+Loading safetensors checkpoint shards:  50% Completed | 8/16 [00:08<00:08,  1.09s/it]
+2026-08-07T08:27:24.927087007Z 
+Loading safetensors checkpoint shards:  56% Completed | 9/16 [00:09<00:08,  1.20s/it]
+2026-08-07T08:27:26.129186864Z 
+Loading safetensors checkpoint shards:  62% Completed | 10/16 [00:11<00:07,  1.20s/it]
+2026-08-07T08:27:27.096541945Z 
+Loading safetensors checkpoint shards:  69% Completed | 11/16 [00:12<00:05,  1.13s/it]
+2026-08-07T08:27:28.423075292Z 
+Loading safetensors checkpoint shards:  75% Completed | 12/16 [00:13<00:04,  1.19s/it]
+2026-08-07T08:27:29.728377839Z 
+Loading safetensors checkpoint shards:  81% Completed | 13/16 [00:14<00:03,  1.22s/it]
+2026-08-07T08:27:30.709363811Z 
+Loading safetensors checkpoint shards:  88% Completed | 14/16 [00:15<00:02,  1.15s/it]
+2026-08-07T08:27:31.781864162Z 
+Loading safetensors checkpoint shards:  94% Completed | 15/16 [00:16<00:01,  1.13s/it]
+2026-08-07T08:27:32.902095577Z 
+Loading safetensors checkpoint shards: 100% Completed | 16/16 [00:17<00:00,  1.12s/it]
+2026-08-07T08:27:32.902243326Z 
+Loading safetensors checkpoint shards: 100% Completed | 16/16 [00:17<00:00,  1.12s/it]
+2026-08-07T08:27:32.902260117Z 
+2026-08-07T08:27:33.346028029Z (VllmWorkerProcess pid=345) INFO 08-07 08:27:33 model_runner.py:1130] Loading model weights took 16.2303 GB
+2026-08-07T08:27:33.346536038Z (VllmWorkerProcess pid=346) INFO 08-07 08:27:33 model_runner.py:1130] Loading model weights took 16.2303 GB
+2026-08-07T08:27:33.346918807Z (VllmWorkerProcess pid=344) INFO 08-07 08:27:33 model_runner.py:1130] Loading model weights took 16.2303 GB
+2026-08-07T08:27:33.357010636Z INFO 08-07 08:27:33 model_runner.py:1130] Loading model weights took 16.2303 GB
+2026-08-07T08:27:55.194463382Z (VllmWorkerProcess pid=344) WARNING 08-07 08:27:55 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 0 (frac=0.9998), replacing with zeros
+2026-08-07T08:27:55.194572513Z (VllmWorkerProcess pid=345) WARNING 08-07 08:27:55 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 0 (frac=0.9998), replacing with zeros
+2026-08-07T08:27:55.194733437Z (VllmWorkerProcess pid=346) WARNING 08-07 08:27:55 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 0 (frac=0.9998), replacing with zeros
+2026-08-07T08:27:55.194804915Z WARNING 08-07 08:27:55 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 0 (frac=0.9998), replacing with zeros
+2026-08-07T08:27:55.196247134Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:55 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:55.196251374Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:55 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:55.196253846Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:55 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:55.196295437Z (VllmWorkerProcess pid=344) WARNING 08-07 08:27:55 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:55.196338474Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:55 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:55.196341519Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:55 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:55.196344149Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:55 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:55.196389650Z (VllmWorkerProcess pid=345) WARNING 08-07 08:27:55 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:55.196529296Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:55 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:55.196533947Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:55 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:55.196536534Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:55 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:55.196583400Z (VllmWorkerProcess pid=346) WARNING 08-07 08:27:55 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:55.196743459Z ERROR 08-07 08:27:55 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:55.196746436Z ERROR 08-07 08:27:55 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:55.196748856Z ERROR 08-07 08:27:55 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:55.196787680Z WARNING 08-07 08:27:55 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:56.898315178Z WARNING 08-07 08:27:56 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 1 (frac=0.9997), replacing with zeros
+2026-08-07T08:27:56.898466780Z (VllmWorkerProcess pid=344) WARNING 08-07 08:27:56 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 1 (frac=0.9997), replacing with zeros
+2026-08-07T08:27:56.898650736Z (VllmWorkerProcess pid=346) WARNING 08-07 08:27:56 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 1 (frac=0.9997), replacing with zeros
+2026-08-07T08:27:56.898655436Z (VllmWorkerProcess pid=345) WARNING 08-07 08:27:56 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 1 (frac=0.9997), replacing with zeros
+2026-08-07T08:27:56.898663860Z ERROR 08-07 08:27:56 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:56.898669821Z ERROR 08-07 08:27:56 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:56.898672533Z ERROR 08-07 08:27:56 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:56.898715835Z WARNING 08-07 08:27:56 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:56.898831930Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:56 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:56.898834864Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:56 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:56.898837465Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:56 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:56.898875999Z (VllmWorkerProcess pid=344) WARNING 08-07 08:27:56 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:56.899004651Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:56 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:56.899007519Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:56 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:56.899010032Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:56 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:56.899049092Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:56 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:56.899051749Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:56 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:56.899053941Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:56 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:56.899056040Z (VllmWorkerProcess pid=346) WARNING 08-07 08:27:56 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:56.899099993Z (VllmWorkerProcess pid=345) WARNING 08-07 08:27:56 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:58.563756628Z WARNING 08-07 08:27:58 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 2 (frac=1.0000), replacing with zeros
+2026-08-07T08:27:58.563839513Z (VllmWorkerProcess pid=346) WARNING 08-07 08:27:58 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 2 (frac=1.0000), replacing with zeros
+2026-08-07T08:27:58.564045316Z (VllmWorkerProcess pid=345) WARNING 08-07 08:27:58 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 2 (frac=1.0000), replacing with zeros
+2026-08-07T08:27:58.564068853Z ERROR 08-07 08:27:58 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:58.564074405Z ERROR 08-07 08:27:58 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:58.564076690Z ERROR 08-07 08:27:58 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:58.564109433Z WARNING 08-07 08:27:58 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:58.564114659Z (VllmWorkerProcess pid=344) WARNING 08-07 08:27:58 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 2 (frac=1.0000), replacing with zeros
+2026-08-07T08:27:58.564153710Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:58 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:58.564166969Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:58 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:58.564169301Z (VllmWorkerProcess pid=346) ERROR 08-07 08:27:58 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:58.564189936Z (VllmWorkerProcess pid=346) WARNING 08-07 08:27:58 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:58.564350334Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:58 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:58.564353987Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:58 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:58.564356275Z (VllmWorkerProcess pid=345) ERROR 08-07 08:27:58 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:58.564393401Z (VllmWorkerProcess pid=345) WARNING 08-07 08:27:58 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:27:58.564416863Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:58 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:27:58.564419370Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:58 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:27:58.564421590Z (VllmWorkerProcess pid=344) ERROR 08-07 08:27:58 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:27:58.564466363Z (VllmWorkerProcess pid=344) WARNING 08-07 08:27:58 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:28:04.108893189Z ERROR 08-07 08:28:04 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:28:04.108904912Z ERROR 08-07 08:28:04 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:28:04.108907541Z ERROR 08-07 08:28:04 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:28:04.108909754Z (VllmWorkerProcess pid=346) ERROR 08-07 08:28:04 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:28:04.108921805Z (VllmWorkerProcess pid=346) ERROR 08-07 08:28:04 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:28:04.108924463Z (VllmWorkerProcess pid=346) ERROR 08-07 08:28:04 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:28:04.108929227Z WARNING 08-07 08:28:04 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:28:04.108932279Z (VllmWorkerProcess pid=346) WARNING 08-07 08:28:04 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:28:04.109043313Z (VllmWorkerProcess pid=345) ERROR 08-07 08:28:04 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:28:04.109063172Z (VllmWorkerProcess pid=345) ERROR 08-07 08:28:04 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:28:04.109066423Z (VllmWorkerProcess pid=345) ERROR 08-07 08:28:04 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:28:04.109068809Z (VllmWorkerProcess pid=344) ERROR 08-07 08:28:04 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:28:04.109071243Z (VllmWorkerProcess pid=344) ERROR 08-07 08:28:04 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:28:04.109073470Z (VllmWorkerProcess pid=344) ERROR 08-07 08:28:04 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:28:04.109096223Z (VllmWorkerProcess pid=345) WARNING 08-07 08:28:04 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:28:04.109107675Z (VllmWorkerProcess pid=344) WARNING 08-07 08:28:04 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:28:05.761670405Z (VllmWorkerProcess pid=346) WARNING 08-07 08:28:05 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 4 (frac=1.0000), replacing with zeros
+2026-08-07T08:28:05.761710885Z WARNING 08-07 08:28:05 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 4 (frac=1.0000), replacing with zeros
+2026-08-07T08:28:05.761782158Z (VllmWorkerProcess pid=344) WARNING 08-07 08:28:05 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 4 (frac=1.0000), replacing with zeros
+2026-08-07T08:28:05.761979258Z (VllmWorkerProcess pid=346) ERROR 08-07 08:28:05 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:28:05.761990343Z (VllmWorkerProcess pid=346) ERROR 08-07 08:28:05 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:28:05.761992582Z (VllmWorkerProcess pid=346) ERROR 08-07 08:28:05 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:28:05.762034982Z (VllmWorkerProcess pid=346) WARNING 08-07 08:28:05 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:28:05.762047237Z (VllmWorkerProcess pid=345) WARNING 08-07 08:28:05 qwen3_5.py:445] NaN in prefill GatedDeltaNet layer 4 (frac=1.0000), replacing with zeros
+2026-08-07T08:28:05.762050526Z ERROR 08-07 08:28:05 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:28:05.762052833Z ERROR 08-07 08:28:05 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:28:05.762056152Z ERROR 08-07 08:28:05 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:28:05.762092143Z WARNING 08-07 08:28:05 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:28:05.762113248Z (VllmWorkerProcess pid=344) ERROR 08-07 08:28:05 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:28:05.762120235Z (VllmWorkerProcess pid=344) ERROR 08-07 08:28:05 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+2026-08-07T08:28:05.762123453Z (VllmWorkerProcess pid=344) ERROR 08-07 08:28:05 _custom_ops.py:58] Please try a clean build and install of vllm,or remove old built files such as vllm/*cpython*.so and build/ .
+2026-08-07T08:28:05.762165494Z (VllmWorkerProcess pid=344) WARNING 08-07 08:28:05 qwen3_5.py:913] FusedMoE native kernel failed (AttributeError: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'), falling back to pure PyTorch experts permanently.
+2026-08-07T08:28:05.762351254Z (VllmWorkerProcess pid=345) ERROR 08-07 08:28:05 _custom_ops.py:58] Error in calling custom op topk_softmax: module 'ixformer.functions' has no attribute 'vllm_moe_topk_softmax'
+2026-08-07T08:28:05.762359500Z (VllmWorkerProcess pid=345) ERROR 08-07 08:28:05 _custom_ops.py:58] Possibly you have built or installed an obsolete version of vllm.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+别人168的docker日志：
+别人168的docker日志：
+别人168的docker日志：
+
+别人168的docker日志：
+别人168的docker日志：
+别人168的docker日志：
+别人168的docker日志：
+
+别人168的docker日志：
+别人168的docker日志：
+别人168的docker日志：
+别人168的docker日志：
+
+别人168的docker日志：
+别人168的docker日志：
+别人168的docker日志：
+别人168的docker日志：
+
+别人168的docker日志：
+
+
+
+2026-07-23T02:28:45.650554047Z /usr/local/corex/lib/python3/dist-packages/torch/cuda/__init__.py:51: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+2026-07-23T02:28:45.650599241Z   import pynvml  # type: ignore[import]
+2026-07-23T02:28:47.200712741Z INFO 07-23 02:28:47 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
+2026-07-23T02:28:48.724165719Z 2026-07-23 02:28:48.724101: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
+2026-07-23T02:28:48.778248291Z 2026-07-23 02:28:48.778192: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
+2026-07-23T02:28:48.778258574Z To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F AVX512_VNNI AVX512_BF16 AVX_VNNI AMX_TILE AMX_INT8 AMX_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
+2026-07-23T02:28:48.811495644Z WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
+2026-07-23T02:28:54.203054296Z INFO 07-23 02:28:54 api_server.py:530] vLLM API server version 0.6.3
+2026-07-23T02:28:54.203282894Z INFO 07-23 02:28:54 api_server.py:531] args: Namespace(host=None, port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=True, enable_auto_tool_choice=True, tool_call_parser='qwen3_coder', tool_parser_plugin='', reasoning_parser='qwen3', model='/model', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=True, download_dir=None, load_format='auto', config_format='auto', dtype='half', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=256000, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=4, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=True, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=0, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.95, num_gpu_blocks_override=None, max_num_batched_tokens=4096, max_num_seqs=2, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=True, max_context_len_to_capture=None, max_seq_len_to_capture=32768, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=True, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['llm'], qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, override_neuron_config=None, scheduling_policy='fcfs', disable_log_requests=True, max_log_len=None, disable_fastapi_docs=False)
+2026-07-23T02:28:54.219648456Z INFO 07-23 02:28:54 config.py:1670] Downcasting torch.float32 to torch.float16.
+2026-07-23T02:29:04.909319894Z WARNING 07-23 02:29:04 arg_utils.py:906] --enable-prefix-caching is currently not supported for multimodal models and has been disabled.
+2026-07-23T02:29:04.909828848Z INFO 07-23 02:29:04 config.py:887] Defaulting to use mp for distributed inference
+2026-07-23T02:29:04.910112374Z INFO 07-23 02:29:04 config.py:1005] Chunked prefill is enabled with max_num_batched_tokens=4096.
+2026-07-23T02:29:04.910210781Z WARNING 07-23 02:29:04 config.py:380] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
+2026-07-23T02:29:04.912872581Z INFO 07-23 02:29:04 llm_engine.py:237] Initializing an LLM engine (v0.6.3) with config: model='/model', speculative_config=None, tokenizer='/model', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=256000, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=4, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=llm, use_v2_block_manager=True, num_scheduler_steps=1, chunked_prefill_enabled=True multi_step_stream_outputs=True, enable_prefix_caching=False, use_async_output_proc=False, use_cached_outputs=False, mm_processor_kwargs=None)
+2026-07-23T02:29:05.406018107Z WARNING 07-23 02:29:05 multiproc_gpu_executor.py:53] Reducing Torch parallelism from 64 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
+2026-07-23T02:29:05.450119566Z INFO 07-23 02:29:05 custom_cache_manager.py:17] Setting Triton cache manager to: vllm.triton_utils.custom_cache_manager:CustomCacheManager
+2026-07-23T02:29:05.500592304Z INFO 07-23 02:29:05 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:05.941321070Z /usr/local/corex/lib/python3/dist-packages/torch/cuda/__init__.py:51: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+2026-07-23T02:29:05.941332229Z   import pynvml  # type: ignore[import]
+2026-07-23T02:29:05.941336669Z /usr/local/corex/lib/python3/dist-packages/torch/cuda/__init__.py:51: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+2026-07-23T02:29:05.941348474Z   import pynvml  # type: ignore[import]
+2026-07-23T02:29:05.941407399Z /usr/local/corex/lib/python3/dist-packages/torch/cuda/__init__.py:51: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+2026-07-23T02:29:05.941412147Z   import pynvml  # type: ignore[import]
+2026-07-23T02:29:07.485947726Z INFO 07-23 02:29:07 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
+2026-07-23T02:29:07.485955803Z INFO 07-23 02:29:07 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
+2026-07-23T02:29:07.485958100Z INFO 07-23 02:29:07 importing.py:10] Triton not installed; certain GPU-related functions will not be available.
+2026-07-23T02:29:09.075001882Z WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
+2026-07-23T02:29:09.075016869Z WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
+2026-07-23T02:29:09.075076452Z WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
+2026-07-23T02:29:14.637614912Z (VllmWorkerProcess pid=345) INFO 07-23 02:29:14 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:14.658248502Z (VllmWorkerProcess pid=345) INFO 07-23 02:29:14 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
+2026-07-23T02:29:14.676296663Z (VllmWorkerProcess pid=346) INFO 07-23 02:29:14 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:14.678468262Z (VllmWorkerProcess pid=344) INFO 07-23 02:29:14 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:14.696698975Z (VllmWorkerProcess pid=346) INFO 07-23 02:29:14 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
+2026-07-23T02:29:14.698898084Z (VllmWorkerProcess pid=344) INFO 07-23 02:29:14 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
+2026-07-23T02:29:15.025099398Z INFO 07-23 02:29:15 shm_broadcast.py:242] vLLM message queue communication handle: Handle(connect_ip='127.0.0.1', local_reader_ranks=[1, 2, 3], buffer=<vllm.distributed.device_communicators.shm_broadcast.ShmRingBuffer object at 0x7f205f9677f0>, local_subscribe_port=41787, remote_subscribe_port=None)
+2026-07-23T02:29:15.071554378Z (VllmWorkerProcess pid=345) INFO 07-23 02:29:15 model_runner.py:1074] Starting to load model /model...
+2026-07-23T02:29:15.071765260Z (VllmWorkerProcess pid=344) INFO 07-23 02:29:15 model_runner.py:1074] Starting to load model /model...
+2026-07-23T02:29:15.071769116Z INFO 07-23 02:29:15 model_runner.py:1074] Starting to load model /model...
+2026-07-23T02:29:15.072467456Z (VllmWorkerProcess pid=346) INFO 07-23 02:29:15 model_runner.py:1074] Starting to load model /model...
+2026-07-23T02:29:15.123205314Z (VllmWorkerProcess pid=344) INFO 07-23 02:29:15 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:15.123647472Z (VllmWorkerProcess pid=346) INFO 07-23 02:29:15 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:15.136250183Z INFO 07-23 02:29:15 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:15.142620520Z (VllmWorkerProcess pid=345) INFO 07-23 02:29:15 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:15.267061728Z 
+Loading safetensors checkpoint shards:   0% Completed | 0/16 [00:00<?, ?it/s]
+2026-07-23T02:29:16.504035279Z 
+Loading safetensors checkpoint shards:   6% Completed | 1/16 [00:01<00:18,  1.24s/it]
+2026-07-23T02:29:17.560515507Z 
+Loading safetensors checkpoint shards:  12% Completed | 2/16 [00:02<00:15,  1.13s/it]
+2026-07-23T02:29:18.454967452Z 
+Loading safetensors checkpoint shards:  19% Completed | 3/16 [00:03<00:13,  1.02s/it]
+2026-07-23T02:29:19.622930900Z 
+Loading safetensors checkpoint shards:  25% Completed | 4/16 [00:04<00:12,  1.08s/it]
+2026-07-23T02:29:20.607005016Z 
+Loading safetensors checkpoint shards:  31% Completed | 5/16 [00:05<00:11,  1.05s/it]
+2026-07-23T02:29:21.663620433Z 
+Loading safetensors checkpoint shards:  38% Completed | 6/16 [00:06<00:10,  1.05s/it]
+2026-07-23T02:29:22.762559585Z 
+Loading safetensors checkpoint shards:  44% Completed | 7/16 [00:07<00:09,  1.07s/it]
+2026-07-23T02:29:23.920513395Z 
+Loading safetensors checkpoint shards:  50% Completed | 8/16 [00:08<00:08,  1.09s/it]
+2026-07-23T02:29:25.327235482Z 
+Loading safetensors checkpoint shards:  56% Completed | 9/16 [00:10<00:08,  1.19s/it]
+2026-07-23T02:29:26.537079790Z 
+Loading safetensors checkpoint shards:  62% Completed | 10/16 [00:11<00:07,  1.20s/it]
+2026-07-23T02:29:27.538590658Z 
+Loading safetensors checkpoint shards:  69% Completed | 11/16 [00:12<00:05,  1.14s/it]
+2026-07-23T02:29:28.909915510Z 
+Loading safetensors checkpoint shards:  75% Completed | 12/16 [00:13<00:04,  1.21s/it]
+2026-07-23T02:29:30.177921768Z 
+Loading safetensors checkpoint shards:  81% Completed | 13/16 [00:14<00:03,  1.23s/it]
+2026-07-23T02:29:31.144529015Z 
+Loading safetensors checkpoint shards:  88% Completed | 14/16 [00:15<00:02,  1.15s/it]
+2026-07-23T02:29:32.170218873Z 
+Loading safetensors checkpoint shards:  94% Completed | 15/16 [00:16<00:01,  1.11s/it]
+2026-07-23T02:29:33.339901076Z 
+Loading safetensors checkpoint shards: 100% Completed | 16/16 [00:18<00:00,  1.13s/it]
+2026-07-23T02:29:33.340059995Z 
+Loading safetensors checkpoint shards: 100% Completed | 16/16 [00:18<00:00,  1.13s/it]
+2026-07-23T02:29:33.340067531Z 
+2026-07-23T02:29:33.791755457Z (VllmWorkerProcess pid=345) INFO 07-23 02:29:33 model_runner.py:1085] Loading model weights took 17.3529 GB
+2026-07-23T02:29:33.791764238Z (VllmWorkerProcess pid=344) INFO 07-23 02:29:33 model_runner.py:1085] Loading model weights took 17.3529 GB
+2026-07-23T02:29:33.792212417Z (VllmWorkerProcess pid=346) INFO 07-23 02:29:33 model_runner.py:1085] Loading model weights took 17.3529 GB
+2026-07-23T02:29:33.799049642Z INFO 07-23 02:29:33 model_runner.py:1085] Loading model weights took 17.3529 GB
+2026-07-23T02:32:25.902136731Z (VllmWorkerProcess pid=346) INFO 07-23 02:32:25 corex_gdn.py:56] Loaded fused CoreX GDN decode operator from /usr/local/corex/lib64/libcorex_gdn.so
+2026-07-23T02:32:25.903895709Z (VllmWorkerProcess pid=345) INFO 07-23 02:32:25 corex_gdn.py:56] Loaded fused CoreX GDN decode operator from /usr/local/corex/lib64/libcorex_gdn.so
+2026-07-23T02:32:25.905723523Z (VllmWorkerProcess pid=346) INFO 07-23 02:32:25 corex_gdn.py:228] Using fused CoreX GDN prefill operator
+2026-07-23T02:32:25.907479743Z (VllmWorkerProcess pid=345) INFO 07-23 02:32:25 corex_gdn.py:228] Using fused CoreX GDN prefill operator
+2026-07-23T02:32:25.915921136Z INFO 07-23 02:32:25 corex_gdn.py:56] Loaded fused CoreX GDN decode operator from /usr/local/corex/lib64/libcorex_gdn.so
+2026-07-23T02:32:25.918179567Z (VllmWorkerProcess pid=344) INFO 07-23 02:32:25 corex_gdn.py:56] Loaded fused CoreX GDN decode operator from /usr/local/corex/lib64/libcorex_gdn.so
+2026-07-23T02:32:25.919624182Z INFO 07-23 02:32:25 corex_gdn.py:228] Using fused CoreX GDN prefill operator
+2026-07-23T02:32:25.921789252Z (VllmWorkerProcess pid=344) INFO 07-23 02:32:25 corex_gdn.py:228] Using fused CoreX GDN prefill operator
+2026-07-23T02:32:25.950838770Z (VllmWorkerProcess pid=346) INFO 07-23 02:32:25 corex_moe.py:339] Using CoreX fused MoE prefill operator: tokens=4096, kernel=expert-grouped-wmma
+2026-07-23T02:32:25.950882505Z (VllmWorkerProcess pid=344) INFO 07-23 02:32:25 corex_moe.py:339] Using CoreX fused MoE prefill operator: tokens=4096, kernel=expert-grouped-wmma
+2026-07-23T02:32:25.950908225Z INFO 07-23 02:32:25 corex_moe.py:339] Using CoreX fused MoE prefill operator: tokens=4096, kernel=expert-grouped-wmma
+2026-07-23T02:32:25.950931658Z (VllmWorkerProcess pid=345) INFO 07-23 02:32:25 corex_moe.py:339] Using CoreX fused MoE prefill operator: tokens=4096, kernel=expert-grouped-wmma
+2026-07-23T02:32:26.045895062Z (VllmWorkerProcess pid=346) INFO 07-23 02:32:26 corex_fa2.py:333] Using CoreX FA2 packed prefill: B=2 Hq=4 Hkv=1 D=256 max_q=2048 max_k=2048
+2026-07-23T02:32:26.045990261Z (VllmWorkerProcess pid=345) INFO 07-23 02:32:26 corex_fa2.py:333] Using CoreX FA2 packed prefill: B=2 Hq=4 Hkv=1 D=256 max_q=2048 max_k=2048
+2026-07-23T02:32:26.046000888Z (VllmWorkerProcess pid=344) INFO 07-23 02:32:26 corex_fa2.py:333] Using CoreX FA2 packed prefill: B=2 Hq=4 Hkv=1 D=256 max_q=2048 max_k=2048
+2026-07-23T02:32:26.046049182Z INFO 07-23 02:32:26 corex_fa2.py:333] Using CoreX FA2 packed prefill: B=2 Hq=4 Hkv=1 D=256 max_q=2048 max_k=2048
+2026-07-23T02:32:27.731056211Z INFO 07-23 02:32:27 distributed_gpu_executor.py:57] # GPU blocks: 19259, # CPU blocks: 6553
+2026-07-23T02:32:27.731106894Z INFO 07-23 02:32:27 distributed_gpu_executor.py:61] Maximum concurrency for 256000 tokens per request: 1.20x
+2026-07-23T02:32:31.498383100Z INFO 07-23 02:32:31 serving_chat.py:107] "auto" tool choice has been enabled please note that while the parallel_tool_calls client option is preset for compatibility reasons, it will be ignored.
+2026-07-23T02:32:31.499707957Z INFO 07-23 02:32:31 serving_chat.py:129] Reasoning parser 'qwen3' enabled.
+2026-07-23T02:32:31.499774658Z WARNING 07-23 02:32:31 serving_embedding.py:199] embedding_mode is False. Embedding API will not work.
+2026-07-23T02:32:31.499825780Z INFO 07-23 02:32:31 launcher.py:19] Available routes are:
+2026-07-23T02:32:31.499867858Z INFO 07-23 02:32:31 launcher.py:27] Route: /openapi.json, Methods: HEAD, GET
+2026-07-23T02:32:31.499886381Z INFO 07-23 02:32:31 launcher.py:27] Route: /docs, Methods: HEAD, GET
+2026-07-23T02:32:31.499904662Z INFO 07-23 02:32:31 launcher.py:27] Route: /docs/oauth2-redirect, Methods: HEAD, GET
+2026-07-23T02:32:31.499945605Z INFO 07-23 02:32:31 launcher.py:27] Route: /redoc, Methods: HEAD, GET
+2026-07-23T02:32:31.499951625Z INFO 07-23 02:32:31 launcher.py:27] Route: /health, Methods: GET
+2026-07-23T02:32:31.499989247Z INFO 07-23 02:32:31 launcher.py:27] Route: /tokenize, Methods: POST
+2026-07-23T02:32:31.499991563Z INFO 07-23 02:32:31 launcher.py:27] Route: /detokenize, Methods: POST
+2026-07-23T02:32:31.500017919Z INFO 07-23 02:32:31 launcher.py:27] Route: /v1/models, Methods: GET
+2026-07-23T02:32:31.500027643Z INFO 07-23 02:32:31 launcher.py:27] Route: /version, Methods: GET
+2026-07-23T02:32:31.500047221Z INFO 07-23 02:32:31 launcher.py:27] Route: /v1/chat/completions, Methods: POST
+2026-07-23T02:32:31.500071390Z INFO 07-23 02:32:31 launcher.py:27] Route: /v1/completions, Methods: POST
+2026-07-23T02:32:31.500084135Z INFO 07-23 02:32:31 launcher.py:27] Route: /v1/embeddings, Methods: POST
+2026-07-23T02:32:31.517724005Z INFO:     Started server process [1]
+2026-07-23T02:32:31.517775355Z INFO:     Waiting for application startup.
+2026-07-23T02:32:31.518104234Z INFO:     Application startup complete.
+2026-07-23T02:32:31.518434320Z INFO:     Uvicorn running on socket ('0.0.0.0', 8000) (Press CTRL+C to quit)
+2026-07-23T02:32:41.518080032Z INFO 07-23 02:32:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:32:44.976380698Z INFO:     172.17.0.1:34848 - "GET /health HTTP/1.1" 200 OK
+2026-07-23T02:32:44.977131890Z INFO:     172.17.0.1:34848 - "GET /v1/models HTTP/1.1" 200 OK
+2026-07-23T02:32:51.517794437Z INFO 07-23 02:32:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:01.518176592Z INFO 07-23 02:33:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:11.517753255Z INFO 07-23 02:33:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:21.518188707Z INFO 07-23 02:33:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:31.518515223Z INFO 07-23 02:33:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:41.518939596Z INFO 07-23 02:33:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:51.519425845Z INFO 07-23 02:33:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:01.519896974Z INFO 07-23 02:34:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:11.520418085Z INFO 07-23 02:34:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:21.520940369Z INFO 07-23 02:34:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:31.521337663Z INFO 07-23 02:34:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:41.521711416Z INFO 07-23 02:34:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:51.522131821Z INFO 07-23 02:34:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:01.522501314Z INFO 07-23 02:35:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:11.522857298Z INFO 07-23 02:35:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:21.523285280Z INFO 07-23 02:35:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:31.522702429Z INFO 07-23 02:35:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:41.523044729Z INFO 07-23 02:35:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:51.523462651Z INFO 07-23 02:35:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:01.523885993Z INFO 07-23 02:36:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:11.524260127Z INFO 07-23 02:36:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:21.523723862Z INFO 07-23 02:36:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:31.524139239Z INFO 07-23 02:36:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:41.523592375Z INFO 07-23 02:36:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:51.523995257Z INFO 07-23 02:36:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:01.524428577Z INFO 07-23 02:37:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:11.525008725Z INFO 07-23 02:37:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:21.524752003Z INFO 07-23 02:37:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:31.525171202Z INFO 07-23 02:37:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:35.774341598Z INFO:     172.28.22.23:22372 - "GET /get_model_info HTTP/1.1" 404 Not Found
+2026-07-23T02:37:35.776361569Z INFO:     172.28.22.23:43360 - "GET /get_server_info HTTP/1.1" 404 Not Found
+2026-07-23T02:37:35.777756478Z INFO:     172.28.22.23:30571 - "GET /version HTTP/1.1" 200 OK
+2026-07-23T02:37:36.257909619Z /usr/local/lib/python3.10/site-packages/pyairports/airports.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
+2026-07-23T02:37:36.257934560Z   from pkg_resources import resource_string
+2026-07-23T02:37:37.889916076Z (VllmWorkerProcess pid=346) INFO 07-23 02:37:37 corex_fa2.py:507] Using CoreX paged FA2 chunked prefill: B=1 Hq=4 Hkv=1 D=256 max_q=17 cache_blocks=2
+2026-07-23T02:37:37.889927964Z INFO 07-23 02:37:37 corex_fa2.py:507] Using CoreX paged FA2 chunked prefill: B=1 Hq=4 Hkv=1 D=256 max_q=17 cache_blocks=2
+2026-07-23T02:37:37.889989104Z (VllmWorkerProcess pid=344) INFO 07-23 02:37:37 corex_fa2.py:507] Using CoreX paged FA2 chunked prefill: B=1 Hq=4 Hkv=1 D=256 max_q=17 cache_blocks=2
+2026-07-23T02:37:37.889993203Z (VllmWorkerProcess pid=345) INFO 07-23 02:37:37 corex_fa2.py:507] Using CoreX paged FA2 chunked prefill: B=1 Hq=4 Hkv=1 D=256 max_q=17 cache_blocks=2
+2026-07-23T02:37:37.970977734Z INFO 07-23 02:37:37 metrics.py:345] Avg prompt throughput: 2.6 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:37.977175350Z (VllmWorkerProcess pid=346) INFO 07-23 02:37:37 corex_gdn.py:138] Using fused CoreX GDN decode operator
+2026-07-23T02:37:37.977199535Z (VllmWorkerProcess pid=344) INFO 07-23 02:37:37 corex_gdn.py:138] Using fused CoreX GDN decode operator
+2026-07-23T02:37:37.977216621Z INFO 07-23 02:37:37 corex_gdn.py:138] Using fused CoreX GDN decode operator
+2026-07-23T02:37:37.977220191Z (VllmWorkerProcess pid=345) INFO 07-23 02:37:37 corex_gdn.py:138] Using fused CoreX GDN decode operator
+2026-07-23T02:37:37.977862426Z (VllmWorkerProcess pid=346) INFO 07-23 02:37:37 corex_moe.py:249] Using CoreX fused MoE decode operator
+2026-07-23T02:37:37.977866130Z (VllmWorkerProcess pid=345) INFO 07-23 02:37:37 corex_moe.py:249] Using CoreX fused MoE decode operator
+2026-07-23T02:37:37.977868711Z INFO 07-23 02:37:37 corex_moe.py:249] Using CoreX fused MoE decode operator
+2026-07-23T02:37:37.977871546Z (VllmWorkerProcess pid=344) INFO 07-23 02:37:37 corex_moe.py:249] Using CoreX fused MoE decode operator
+2026-07-23T02:37:43.000160753Z INFO 07-23 02:37:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:44.269161765Z INFO:     172.28.22.23:13116 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:37:44.396724776Z INFO:     172.28.22.23:7057 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:37:48.010705708Z INFO 07-23 02:37:48 metrics.py:345] Avg prompt throughput: 57.9 tokens/s, Avg generation throughput: 16.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:49.136976286Z INFO:     172.28.22.23:45259 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:37:53.025290553Z INFO 07-23 02:37:53 metrics.py:345] Avg prompt throughput: 7.2 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:58.061514748Z INFO 07-23 02:37:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:03.073788209Z INFO 07-23 02:38:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:06.918087930Z INFO:     172.28.22.23:34227 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:38:08.377827090Z INFO 07-23 02:38:08 metrics.py:345] Avg prompt throughput: 4.3 tokens/s, Avg generation throughput: 15.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:13.046591181Z INFO:     172.28.22.23:29692 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:38:13.172739732Z INFO:     172.28.22.23:48540 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:38:13.710792259Z INFO 07-23 02:38:13 metrics.py:345] Avg prompt throughput: 58.3 tokens/s, Avg generation throughput: 18.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:14.666143261Z INFO:     172.28.22.23:34379 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:38:18.720975352Z INFO 07-23 02:38:18 metrics.py:345] Avg prompt throughput: 74.2 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:23.725539749Z INFO 07-23 02:38:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:28.760348109Z INFO 07-23 02:38:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:33.763160416Z INFO 07-23 02:38:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:38.786603579Z INFO 07-23 02:38:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:43.822945213Z INFO 07-23 02:38:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:48.829240975Z INFO 07-23 02:38:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:53.861865865Z INFO 07-23 02:38:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:58.900819345Z INFO 07-23 02:38:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:03.930669117Z INFO 07-23 02:39:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:08.932255487Z INFO 07-23 02:39:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:13.947463156Z INFO 07-23 02:39:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:17.129519291Z INFO:     172.28.22.23:47285 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:21.526611923Z INFO 07-23 02:39:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 8.7 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:22.789790337Z ERROR 07-23 02:39:22 serving_chat.py:227] Error in loading multi-modal data
+2026-07-23T02:39:22.789821427Z ERROR 07-23 02:39:22 serving_chat.py:227] Traceback (most recent call last):
+2026-07-23T02:39:22.789824505Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 770, in _request
+2026-07-23T02:39:22.789827021Z ERROR 07-23 02:39:22 serving_chat.py:227]     resp = await handler(req)
+2026-07-23T02:39:22.789829231Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 725, in _connect_and_send_request
+2026-07-23T02:39:22.789831554Z ERROR 07-23 02:39:22 serving_chat.py:227]     conn = await self._connector.connect(
+2026-07-23T02:39:22.789833480Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 642, in connect
+2026-07-23T02:39:22.789835993Z ERROR 07-23 02:39:22 serving_chat.py:227]     proto = await self._create_connection(req, traces, timeout)
+2026-07-23T02:39:22.789838507Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1209, in _create_connection
+2026-07-23T02:39:22.789840743Z ERROR 07-23 02:39:22 serving_chat.py:227]     _, proto = await self._create_direct_connection(req, traces, timeout)
+2026-07-23T02:39:22.789842704Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1550, in _create_direct_connection
+2026-07-23T02:39:22.789844833Z ERROR 07-23 02:39:22 serving_chat.py:227]     transp, proto = await self._wrap_create_connection(
+2026-07-23T02:39:22.789857869Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1283, in _wrap_create_connection
+2026-07-23T02:39:22.789859986Z ERROR 07-23 02:39:22 serving_chat.py:227]     return await self._loop.create_connection(*args, **kwargs, sock=sock)
+2026-07-23T02:39:22.789861916Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "uvloop/loop.pyx", line 2088, in create_connection
+2026-07-23T02:39:22.789864184Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "uvloop/loop.pyx", line 2083, in uvloop.loop.Loop.create_connection
+2026-07-23T02:39:22.789866163Z ERROR 07-23 02:39:22 serving_chat.py:227] asyncio.exceptions.CancelledError
+2026-07-23T02:39:22.789867986Z ERROR 07-23 02:39:22 serving_chat.py:227] 
+2026-07-23T02:39:22.789878123Z ERROR 07-23 02:39:22 serving_chat.py:227] The above exception was the direct cause of the following exception:
+2026-07-23T02:39:22.789887569Z ERROR 07-23 02:39:22 serving_chat.py:227] 
+2026-07-23T02:39:22.789890593Z ERROR 07-23 02:39:22 serving_chat.py:227] Traceback (most recent call last):
+2026-07-23T02:39:22.789892686Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 225, in create_chat_completion
+2026-07-23T02:39:22.789894923Z ERROR 07-23 02:39:22 serving_chat.py:227]     mm_data = await mm_data_future
+2026-07-23T02:39:22.789896983Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 248, in all_mm_data
+2026-07-23T02:39:22.789899260Z ERROR 07-23 02:39:22 serving_chat.py:227]     items = await asyncio.gather(*self._items)
+2026-07-23T02:39:22.789901151Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/multimodal/utils.py", line 140, in async_get_and_parse_image
+2026-07-23T02:39:22.789903256Z ERROR 07-23 02:39:22 serving_chat.py:227]     image = await async_fetch_image(image_url)
+2026-07-23T02:39:22.789905277Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/multimodal/utils.py", line 62, in async_fetch_image
+2026-07-23T02:39:22.789907335Z ERROR 07-23 02:39:22 serving_chat.py:227]     image_raw = await global_http_connection.async_get_bytes(
+2026-07-23T02:39:22.789909226Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/connections.py", line 92, in async_get_bytes
+2026-07-23T02:39:22.789911211Z ERROR 07-23 02:39:22 serving_chat.py:227]     async with await self.get_async_response(url, timeout=timeout) as r:
+2026-07-23T02:39:22.789913906Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 1488, in __aenter__
+2026-07-23T02:39:22.789915849Z ERROR 07-23 02:39:22 serving_chat.py:227]     self._resp: _RetType = await self._coro
+2026-07-23T02:39:22.789918323Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 622, in _request
+2026-07-23T02:39:22.789921398Z ERROR 07-23 02:39:22 serving_chat.py:227]     with timer:
+2026-07-23T02:39:22.789929127Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/helpers.py", line 685, in __exit__
+2026-07-23T02:39:22.789931189Z ERROR 07-23 02:39:22 serving_chat.py:227]     raise asyncio.TimeoutError from exc_val
+2026-07-23T02:39:22.789933347Z ERROR 07-23 02:39:22 serving_chat.py:227] asyncio.exceptions.TimeoutError
+2026-07-23T02:39:22.789982206Z INFO:     172.28.22.23:63905 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:39:23.339790176Z INFO:     172.28.22.23:48915 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:23.862218623Z INFO:     172.28.22.23:39188 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:26.537243563Z INFO 07-23 02:39:26 metrics.py:345] Avg prompt throughput: 12.0 tokens/s, Avg generation throughput: 12.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:31.709534191Z INFO:     172.28.22.23:30519 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:31.709975900Z INFO 07-23 02:39:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:32.540087790Z INFO:     172.28.22.23:28182 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:36.711389406Z INFO 07-23 02:39:36 metrics.py:345] Avg prompt throughput: 6.0 tokens/s, Avg generation throughput: 20.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:38.834910565Z INFO:     172.28.22.23:25497 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:40.106985905Z INFO:     172.28.22.23:31187 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:41.375383434Z INFO:     172.28.22.23:1054 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:41.736972408Z INFO 07-23 02:39:41 metrics.py:345] Avg prompt throughput: 6.6 tokens/s, Avg generation throughput: 17.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:42.644170272Z INFO:     172.28.22.23:10283 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:43.909888974Z INFO:     172.28.22.23:43195 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:44.026597768Z INFO:     172.28.22.23:31826 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:39:45.279682681Z INFO:     172.28.22.23:31890 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:46.532238457Z INFO:     172.28.22.23:52470 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:46.768052149Z INFO 07-23 02:39:46 metrics.py:345] Avg prompt throughput: 8.7 tokens/s, Avg generation throughput: 15.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:47.779232275Z INFO:     172.28.22.23:1935 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:47.894850660Z INFO:     172.28.22.23:24799 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:39:49.140115276Z INFO:     172.28.22.23:32610 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:50.385833050Z INFO:     172.28.22.23:34946 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:51.632711881Z INFO:     172.28.22.23:29640 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:51.824060918Z INFO 07-23 02:39:51 metrics.py:345] Avg prompt throughput: 8.7 tokens/s, Avg generation throughput: 15.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:52.883677883Z INFO:     172.28.22.23:25348 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:54.134965370Z INFO:     172.28.22.23:37574 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:55.381846551Z INFO:     172.28.22.23:9523 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:56.629565470Z INFO:     172.28.22.23:56394 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:56.827523945Z INFO 07-23 02:39:56 metrics.py:345] Avg prompt throughput: 8.8 tokens/s, Avg generation throughput: 16.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:58.129748260Z INFO:     172.28.22.23:9093 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:40:01.848026951Z INFO 07-23 02:40:01 metrics.py:345] Avg prompt throughput: 2.8 tokens/s, Avg generation throughput: 23.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:06.858910882Z INFO 07-23 02:40:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:11.862489179Z INFO 07-23 02:40:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:16.899163580Z INFO 07-23 02:40:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:21.910058417Z INFO 07-23 02:40:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:26.929209213Z INFO 07-23 02:40:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:31.962337057Z INFO 07-23 02:40:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:36.964040774Z INFO 07-23 02:40:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:41.993035873Z INFO 07-23 02:40:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:43.624005510Z INFO:     172.28.22.23:2587 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:40:44.061237847Z INFO:     172.28.22.23:6084 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:40:47.185504184Z INFO:     172.28.22.23:50569 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:40:47.377278073Z INFO 07-23 02:40:47 metrics.py:345] Avg prompt throughput: 7.8 tokens/s, Avg generation throughput: 17.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:52.412861124Z INFO 07-23 02:40:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:57.415707222Z INFO 07-23 02:40:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:02.437584264Z INFO 07-23 02:41:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:07.476637249Z INFO 07-23 02:41:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:12.517176365Z INFO 07-23 02:41:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:17.531413276Z INFO 07-23 02:41:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:22.564441104Z INFO 07-23 02:41:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:27.577151702Z INFO 07-23 02:41:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:32.611584807Z INFO 07-23 02:41:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:37.623284038Z INFO 07-23 02:41:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:42.639031894Z INFO 07-23 02:41:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:47.641862259Z INFO 07-23 02:41:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:52.683325904Z INFO 07-23 02:41:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:57.720443718Z INFO 07-23 02:41:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:02.748390108Z INFO 07-23 02:42:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:07.784213915Z INFO 07-23 02:42:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:12.827245969Z INFO 07-23 02:42:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:17.859544046Z INFO 07-23 02:42:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:22.889429150Z INFO 07-23 02:42:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:27.923212049Z INFO 07-23 02:42:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:32.925156292Z INFO 07-23 02:42:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:37.955994920Z INFO 07-23 02:42:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:40.293610439Z INFO:     172.28.22.23:17026 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:40.411301190Z INFO:     172.28.22.23:47159 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:42:40.524865477Z INFO:     172.28.22.23:53783 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:42:40.637743018Z INFO:     172.28.22.23:24159 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:42:41.181611635Z INFO:     172.28.22.23:38467 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:41.727889239Z INFO:     172.28.22.23:24963 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:42.962296089Z INFO 07-23 02:42:42 metrics.py:345] Avg prompt throughput: 64.9 tokens/s, Avg generation throughput: 14.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:43.783197309Z INFO:     172.28.22.23:14284 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:44.459669799Z INFO:     172.28.22.23:49964 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:44.575803355Z INFO:     172.28.22.23:54360 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:47.973780549Z INFO 07-23 02:42:47 metrics.py:345] Avg prompt throughput: 15.0 tokens/s, Avg generation throughput: 18.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:01.858799064Z INFO 07-23 02:43:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:10.553432764Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:04<00:09,  4.87s/it]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:04<00:00,  1.31s/it]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:05<00:00,  1.69s/it]
+2026-07-23T02:43:10.569991401Z INFO 07-23 02:43:10 metrics.py:345] Avg prompt throughput: 4.0 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:11.990491527Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.23it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.17it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  6.98it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.52it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.10it/s]
+2026-07-23T02:43:13.445065774Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.22it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.22it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  7.07it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.63it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.14it/s]
+2026-07-23T02:43:14.876448068Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.23it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.28it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  7.15it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.73it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.23it/s]
+2026-07-23T02:43:15.701717222Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.19it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.19it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.69it/s]
+2026-07-23T02:43:15.720137025Z INFO 07-23 02:43:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:16.473274583Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.21it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  5.71it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.28it/s]
+2026-07-23T02:43:18.904638207Z 
+Compiling FSM index for all state transitions:   0%|          | 0/23 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   4%|▍         | 1/23 [00:00<00:10,  2.14it/s]
+Compiling FSM index for all state transitions:  13%|█▎        | 3/23 [00:00<00:04,  4.13it/s]
+Compiling FSM index for all state transitions:  22%|██▏       | 5/23 [00:00<00:02,  6.94it/s]
+Compiling FSM index for all state transitions:  30%|███       | 7/23 [00:00<00:01,  9.55it/s]
+Compiling FSM index for all state transitions:  39%|███▉      | 9/23 [00:01<00:01, 11.88it/s]
+Compiling FSM index for all state transitions:  48%|████▊     | 11/23 [00:01<00:00, 13.84it/s]
+Compiling FSM index for all state transitions:  57%|█████▋    | 13/23 [00:01<00:00, 15.41it/s]
+Compiling FSM index for all state transitions:  65%|██████▌   | 15/23 [00:01<00:00, 16.62it/s]
+Compiling FSM index for all state transitions:  74%|███████▍  | 17/23 [00:01<00:00, 17.54it/s]
+Compiling FSM index for all state transitions:  87%|████████▋ | 20/23 [00:01<00:00, 18.47it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:01<00:00, 19.06it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:02<00:00,  9.89it/s]
+2026-07-23T02:43:21.350197196Z 
+Compiling FSM index for all state transitions:   0%|          | 0/23 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   4%|▍         | 1/23 [00:00<00:10,  2.14it/s]
+Compiling FSM index for all state transitions:  13%|█▎        | 3/23 [00:00<00:04,  4.24it/s]
+Compiling FSM index for all state transitions:  22%|██▏       | 5/23 [00:00<00:02,  7.05it/s]
+Compiling FSM index for all state transitions:  30%|███       | 7/23 [00:00<00:01,  9.67it/s]
+Compiling FSM index for all state transitions:  39%|███▉      | 9/23 [00:01<00:01, 11.99it/s]
+Compiling FSM index for all state transitions:  52%|█████▏    | 12/23 [00:01<00:00, 14.60it/s]
+Compiling FSM index for all state transitions:  61%|██████    | 14/23 [00:01<00:00, 15.87it/s]
+Compiling FSM index for all state transitions:  74%|███████▍  | 17/23 [00:01<00:00, 17.26it/s]
+Compiling FSM index for all state transitions:  87%|████████▋ | 20/23 [00:01<00:00, 18.16it/s]
+Compiling FSM index for all state transitions:  96%|█████████▌| 22/23 [00:01<00:00, 18.40it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:02<00:00,  9.89it/s]
+2026-07-23T02:43:21.397139672Z INFO 07-23 02:43:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:22.261063391Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.22it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.24it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.73it/s]
+2026-07-23T02:43:22.957955678Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.25it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.31it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.77it/s]
+2026-07-23T02:43:24.351851677Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.25it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.32it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  7.21it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.80it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.26it/s]
+2026-07-23T02:43:25.800354211Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.16it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.28it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  7.15it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.73it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.16it/s]
+2026-07-23T02:43:26.709113498Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.23it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.25it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.73it/s]
+2026-07-23T02:43:26.727308641Z INFO 07-23 02:43:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:27.404963736Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.25it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.30it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.78it/s]
+2026-07-23T02:43:29.815689817Z 
+Compiling FSM index for all state transitions:   0%|          | 0/23 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   4%|▍         | 1/23 [00:00<00:10,  2.14it/s]
+Compiling FSM index for all state transitions:  13%|█▎        | 3/23 [00:00<00:04,  4.21it/s]
+Compiling FSM index for all state transitions:  22%|██▏       | 5/23 [00:00<00:02,  7.04it/s]
+Compiling FSM index for all state transitions:  30%|███       | 7/23 [00:00<00:01,  9.64it/s]
+Compiling FSM index for all state transitions:  39%|███▉      | 9/23 [00:01<00:01, 11.95it/s]
+Compiling FSM index for all state transitions:  48%|████▊     | 11/23 [00:01<00:00, 13.88it/s]
+Compiling FSM index for all state transitions:  57%|█████▋    | 13/23 [00:01<00:00, 15.42it/s]
+Compiling FSM index for all state transitions:  65%|██████▌   | 15/23 [00:01<00:00, 16.60it/s]
+Compiling FSM index for all state transitions:  74%|███████▍  | 17/23 [00:01<00:00, 17.53it/s]
+Compiling FSM index for all state transitions:  87%|████████▋ | 20/23 [00:01<00:00, 18.46it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:01<00:00, 18.81it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:02<00:00,  9.91it/s]
+2026-07-23T02:43:31.881017891Z INFO 07-23 02:43:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:32.258419554Z 
+Compiling FSM index for all state transitions:   0%|          | 0/23 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   4%|▍         | 1/23 [00:00<00:10,  2.13it/s]
+Compiling FSM index for all state transitions:  13%|█▎        | 3/23 [00:00<00:04,  4.25it/s]
+Compiling FSM index for all state transitions:  22%|██▏       | 5/23 [00:00<00:02,  7.10it/s]
+Compiling FSM index for all state transitions:  30%|███       | 7/23 [00:00<00:01,  9.72it/s]
+Compiling FSM index for all state transitions:  39%|███▉      | 9/23 [00:01<00:01, 12.03it/s]
+Compiling FSM index for all state transitions:  48%|████▊     | 11/23 [00:01<00:00, 13.97it/s]
+Compiling FSM index for all state transitions:  57%|█████▋    | 13/23 [00:01<00:00, 15.50it/s]
+Compiling FSM index for all state transitions:  65%|██████▌   | 15/23 [00:01<00:00, 16.69it/s]
+Compiling FSM index for all state transitions:  74%|███████▍  | 17/23 [00:01<00:00, 17.58it/s]
+Compiling FSM index for all state transitions:  83%|████████▎ | 19/23 [00:01<00:00, 18.25it/s]
+Compiling FSM index for all state transitions:  96%|█████████▌| 22/23 [00:01<00:00, 18.93it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:02<00:00,  9.89it/s]
+2026-07-23T02:43:32.983539758Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.24it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.29it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.76it/s]
+2026-07-23T02:43:33.672113037Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.25it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.32it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.78it/s]
+2026-07-23T02:43:34.368060101Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.24it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.28it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.76it/s]
+2026-07-23T02:43:34.941584523Z 
+Compiling FSM index for all state transitions:   0%|          | 0/2 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  50%|█████     | 1/2 [00:00<00:00,  2.25it/s]
+Compiling FSM index for all state transitions:  50%|█████     | 1/2 [00:00<00:00,  1.96it/s]
+2026-07-23T02:43:35.230436592Z INFO:     172.28.22.23:18839 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:38.310835204Z 
+Compiling FSM index for all state transitions:   0%|          | 0/35 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   3%|▎         | 1/35 [00:00<00:16,  2.11it/s]
+Compiling FSM index for all state transitions:   9%|▊         | 3/35 [00:00<00:05,  6.13it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 5/35 [00:00<00:03,  9.43it/s]
+Compiling FSM index for all state transitions:  20%|██        | 7/35 [00:00<00:02, 12.06it/s]
+Compiling FSM index for all state transitions:  26%|██▌       | 9/35 [00:00<00:01, 13.89it/s]
+Compiling FSM index for all state transitions:  31%|███▏      | 11/35 [00:01<00:01, 15.18it/s]
+Compiling FSM index for all state transitions:  37%|███▋      | 13/35 [00:01<00:02, 10.52it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 15/35 [00:01<00:01, 12.25it/s]
+Compiling FSM index for all state transitions:  49%|████▊     | 17/35 [00:01<00:01, 13.80it/s]
+Compiling FSM index for all state transitions:  54%|█████▍    | 19/35 [00:01<00:01, 14.75it/s]
+Compiling FSM index for all state transitions:  60%|██████    | 21/35 [00:01<00:00, 15.66it/s]
+Compiling FSM index for all state transitions:  66%|██████▌   | 23/35 [00:01<00:00, 16.67it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 25/35 [00:01<00:00, 17.44it/s]
+Compiling FSM index for all state transitions:  77%|███████▋  | 27/35 [00:02<00:00, 17.73it/s]
+Compiling FSM index for all state transitions:  83%|████████▎ | 29/35 [00:02<00:00, 17.93it/s]
+Compiling FSM index for all state transitions:  89%|████████▊ | 31/35 [00:02<00:00, 18.39it/s]
+Compiling FSM index for all state transitions:  94%|█████████▍| 33/35 [00:02<00:00, 18.07it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 35/35 [00:02<00:00, 18.50it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 35/35 [00:02<00:00, 11.94it/s]
+2026-07-23T02:43:38.462680310Z INFO 07-23 02:43:38 metrics.py:345] Avg prompt throughput: 3.5 tokens/s, Avg generation throughput: 0.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:39.428082640Z INFO:     172.28.22.23:41311 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:41.354629107Z INFO:     172.28.22.23:35141 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:41.951932899Z INFO:     172.28.22.23:40937 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:42.708616569Z INFO:     172.28.22.23:3677 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:43.378056127Z INFO:     172.28.22.23:46863 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:43.593932801Z INFO 07-23 02:43:43 metrics.py:345] Avg prompt throughput: 22.6 tokens/s, Avg generation throughput: 13.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:44.482046916Z INFO:     172.28.22.23:46268 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:51.952117180Z INFO 07-23 02:43:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:01.952434751Z INFO 07-23 02:44:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:11.953116612Z INFO 07-23 02:44:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:21.953465417Z INFO 07-23 02:44:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:31.954404947Z INFO 07-23 02:44:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:41.955602923Z INFO 07-23 02:44:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:51.956600553Z INFO 07-23 02:44:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:01.429424131Z INFO 07-23 02:45:01 metrics.py:345] Avg prompt throughput: 33.4 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:02.020271472Z INFO:     172.28.22.23:10993 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:45:02.023173066Z INFO:     172.28.22.23:34691 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:45:02.542033396Z INFO:     172.28.22.23:44245 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:45:03.031598393Z INFO:     172.28.22.23:5495 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:45:03.034079308Z INFO:     172.28.22.23:33361 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:45:03.035816648Z INFO:     172.28.22.23:1279 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:45:03.038855639Z ERROR 07-23 02:45:03 serving_chat.py:221] Error in applying chat template from request
+2026-07-23T02:45:03.038865861Z ERROR 07-23 02:45:03 serving_chat.py:221] Traceback (most recent call last):
+2026-07-23T02:45:03.038868246Z ERROR 07-23 02:45:03 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 210, in create_chat_completion
+2026-07-23T02:45:03.038870615Z ERROR 07-23 02:45:03 serving_chat.py:221]     prompt = apply_hf_chat_template(
+2026-07-23T02:45:03.038872879Z ERROR 07-23 02:45:03 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 574, in apply_hf_chat_template
+2026-07-23T02:45:03.038875232Z ERROR 07-23 02:45:03 serving_chat.py:221]     return tokenizer.apply_chat_template(
+2026-07-23T02:45:03.038877096Z ERROR 07-23 02:45:03 serving_chat.py:221]   File "/usr/local/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1623, in apply_chat_template
+2026-07-23T02:45:03.038879145Z ERROR 07-23 02:45:03 serving_chat.py:221]     isinstance(conversation[0], (list, tuple)) or hasattr(conversation[0], "messages")
+2026-07-23T02:45:03.038881206Z ERROR 07-23 02:45:03 serving_chat.py:221] IndexError: list index out of range
+2026-07-23T02:45:03.039113159Z /usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/api_server.py:316: RuntimeWarning: coroutine 'AsyncMultiModalItemTracker.all_mm_data' was never awaited
+2026-07-23T02:45:03.039116978Z   generator = await chat(raw_request).create_chat_completion(
+2026-07-23T02:45:03.039119691Z RuntimeWarning: Enable tracemalloc to get the object allocation traceback
+2026-07-23T02:45:03.039249660Z INFO:     172.28.22.23:39148 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:45:12.020995535Z INFO 07-23 02:45:12 metrics.py:345] Avg prompt throughput: 3.6 tokens/s, Avg generation throughput: 1.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:18.489898901Z INFO:     172.28.22.23:47283 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:45:18.707020154Z INFO 07-23 02:45:18 metrics.py:345] Avg prompt throughput: 5.8 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:23.747292153Z INFO 07-23 02:45:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:28.783886861Z INFO 07-23 02:45:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:33.805881768Z INFO 07-23 02:45:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:38.828813167Z INFO 07-23 02:45:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:43.837844599Z INFO 07-23 02:45:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:48.863019622Z INFO 07-23 02:45:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:53.902955303Z INFO 07-23 02:45:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:58.919192631Z INFO 07-23 02:45:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:03.934701110Z INFO 07-23 02:46:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:08.965144483Z INFO 07-23 02:46:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:13.969498143Z INFO 07-23 02:46:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:19.000136074Z INFO 07-23 02:46:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:24.035080024Z INFO 07-23 02:46:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:29.054704172Z INFO 07-23 02:46:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:34.075751226Z INFO 07-23 02:46:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:39.109012197Z INFO 07-23 02:46:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:44.142536722Z INFO 07-23 02:46:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:49.151030740Z INFO 07-23 02:46:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:54.155952934Z INFO 07-23 02:46:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:59.168320199Z INFO 07-23 02:46:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:04.198850701Z INFO 07-23 02:47:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:09.227769526Z INFO 07-23 02:47:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:14.236849600Z INFO 07-23 02:47:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:19.250060510Z INFO 07-23 02:47:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:24.278224796Z INFO 07-23 02:47:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:29.302744374Z INFO 07-23 02:47:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:34.340169258Z INFO 07-23 02:47:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:39.373263942Z INFO 07-23 02:47:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:44.400614703Z INFO 07-23 02:47:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:49.429303585Z INFO 07-23 02:47:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:54.454631971Z INFO 07-23 02:47:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:59.476270315Z INFO 07-23 02:47:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:04.489781001Z INFO 07-23 02:48:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:09.502845316Z INFO 07-23 02:48:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:14.525253044Z INFO 07-23 02:48:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:19.535460002Z INFO 07-23 02:48:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:24.549336142Z INFO 07-23 02:48:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:29.562276558Z INFO 07-23 02:48:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:34.582693092Z INFO 07-23 02:48:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:39.606402177Z INFO 07-23 02:48:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:44.618006935Z INFO 07-23 02:48:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:49.639887326Z INFO 07-23 02:48:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:54.661553164Z INFO 07-23 02:48:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:59.691872658Z INFO 07-23 02:48:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:04.716442787Z INFO 07-23 02:49:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:09.737600900Z INFO 07-23 02:49:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:14.765495628Z INFO 07-23 02:49:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:19.785966586Z INFO 07-23 02:49:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:24.808905236Z INFO 07-23 02:49:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:29.823237965Z INFO 07-23 02:49:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:34.840213253Z INFO 07-23 02:49:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:39.860991050Z INFO 07-23 02:49:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:44.886401826Z INFO 07-23 02:49:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:49.902498154Z INFO 07-23 02:49:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:54.932337177Z INFO 07-23 02:49:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:59.972912007Z INFO 07-23 02:49:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:05.007985393Z INFO 07-23 02:50:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:10.048282976Z INFO 07-23 02:50:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:15.051923596Z INFO 07-23 02:50:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:20.093869860Z INFO 07-23 02:50:20 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:25.125545341Z INFO 07-23 02:50:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:30.161930016Z INFO 07-23 02:50:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:35.204345197Z INFO 07-23 02:50:35 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:40.208651498Z INFO 07-23 02:50:40 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:45.211397957Z INFO 07-23 02:50:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:50.215514007Z INFO 07-23 02:50:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:55.226202583Z INFO 07-23 02:50:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:00.234263458Z INFO 07-23 02:51:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:05.268080221Z INFO 07-23 02:51:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:10.295027960Z INFO 07-23 02:51:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:15.333710639Z INFO 07-23 02:51:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:22.023882201Z INFO 07-23 02:51:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 6.1 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:32.024428280Z INFO 07-23 02:51:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:42.025051809Z INFO 07-23 02:51:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:52.025669180Z INFO 07-23 02:51:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:02.026245267Z INFO 07-23 02:52:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:12.025670280Z INFO 07-23 02:52:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:22.025991502Z INFO 07-23 02:52:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:32.026096441Z INFO 07-23 02:52:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:42.026445024Z INFO 07-23 02:52:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:52.026881786Z INFO 07-23 02:52:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:02.027443741Z INFO 07-23 02:53:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:12.027857295Z INFO 07-23 02:53:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:22.021717002Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:22.028015831Z INFO 07-23 02:53:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:22.034636325Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:22.042292538Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:22.049601298Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:22.237627231Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:53:27.059387000Z INFO 07-23 02:53:27 metrics.py:345] Avg prompt throughput: 1478.3 tokens/s, Avg generation throughput: 1.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:32.069675497Z INFO 07-23 02:53:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:37.075811510Z INFO 07-23 02:53:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:41.384843095Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:41.531870163Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:53:42.183280911Z INFO 07-23 02:53:42 metrics.py:345] Avg prompt throughput: 179.7 tokens/s, Avg generation throughput: 19.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:47.184870989Z INFO 07-23 02:53:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:52.202464890Z INFO 07-23 02:53:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:57.216053216Z INFO 07-23 02:53:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:02.236765351Z INFO 07-23 02:54:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:07.245326134Z INFO 07-23 02:54:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:12.257378000Z INFO 07-23 02:54:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:17.265723592Z INFO 07-23 02:54:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:22.274426444Z INFO 07-23 02:54:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:27.288180475Z INFO 07-23 02:54:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:32.303355245Z INFO 07-23 02:54:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:37.308955789Z INFO 07-23 02:54:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:42.316230767Z INFO 07-23 02:54:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:47.330731343Z INFO 07-23 02:54:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:52.350938716Z INFO 07-23 02:54:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:57.368955403Z INFO 07-23 02:54:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:02.387972119Z INFO 07-23 02:55:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:07.403238913Z INFO 07-23 02:55:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:12.424487516Z INFO 07-23 02:55:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:17.468209611Z INFO 07-23 02:55:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:22.468838506Z INFO 07-23 02:55:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:27.495305182Z INFO 07-23 02:55:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:31.663705730Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:55:33.559722894Z INFO 07-23 02:55:33 metrics.py:345] Avg prompt throughput: 675.4 tokens/s, Avg generation throughput: 14.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:39.017972246Z INFO 07-23 02:55:39 metrics.py:345] Avg prompt throughput: 2251.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:45.229906291Z INFO 07-23 02:55:45 metrics.py:345] Avg prompt throughput: 1978.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:52.031344888Z INFO 07-23 02:55:52 metrics.py:345] Avg prompt throughput: 1204.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:56.488143481Z (VllmWorkerProcess pid=346) INFO 07-23 02:55:56 corex_fa2.py:225] Using CoreX paged decode: B=1 Hq=4 Hkv=1 D=256 max_k=45455 partition=256
+2026-07-23T02:55:56.488439965Z INFO 07-23 02:55:56 corex_fa2.py:225] Using CoreX paged decode: B=1 Hq=4 Hkv=1 D=256 max_k=45455 partition=256
+2026-07-23T02:55:56.488465183Z (VllmWorkerProcess pid=345) INFO 07-23 02:55:56 corex_fa2.py:225] Using CoreX paged decode: B=1 Hq=4 Hkv=1 D=256 max_k=45455 partition=256
+2026-07-23T02:55:56.488468721Z (VllmWorkerProcess pid=344) INFO 07-23 02:55:56 corex_fa2.py:225] Using CoreX paged decode: B=1 Hq=4 Hkv=1 D=256 max_k=45455 partition=256
+2026-07-23T02:55:57.074355971Z INFO 07-23 02:55:57 metrics.py:345] Avg prompt throughput: 1703.4 tokens/s, Avg generation throughput: 2.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:02.075744305Z INFO 07-23 02:56:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:04.960717007Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:05.133613133Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:56:07.092301075Z INFO 07-23 02:56:07 metrics.py:345] Avg prompt throughput: 664.6 tokens/s, Avg generation throughput: 12.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:12.126688361Z INFO 07-23 02:56:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:14.381995529Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:14.389331161Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:14.395208292Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:14.402701762Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:14.562022048Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:56:18.164082686Z INFO 07-23 02:56:18 metrics.py:345] Avg prompt throughput: 1356.9 tokens/s, Avg generation throughput: 8.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:23.187795865Z INFO 07-23 02:56:23 metrics.py:345] Avg prompt throughput: 620.6 tokens/s, Avg generation throughput: 7.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:28.216477560Z INFO 07-23 02:56:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:32.311129833Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:32.500362238Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:56:34.377488148Z INFO 07-23 02:56:34 metrics.py:345] Avg prompt throughput: 664.8 tokens/s, Avg generation throughput: 14.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:39.826490851Z INFO 07-23 02:56:39 metrics.py:345] Avg prompt throughput: 2255.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:44.855411293Z INFO 07-23 02:56:44 metrics.py:345] Avg prompt throughput: 1032.6 tokens/s, Avg generation throughput: 2.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:49.884515182Z INFO 07-23 02:56:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:54.030501607Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:56:55.903654883Z INFO 07-23 02:56:55 metrics.py:345] Avg prompt throughput: 680.5 tokens/s, Avg generation throughput: 14.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:01.492620867Z INFO 07-23 02:57:01 metrics.py:345] Avg prompt throughput: 1348.2 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:06.518263412Z INFO 07-23 02:57:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:06.756437999Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:57:12.034175081Z INFO 07-23 02:57:12 metrics.py:345] Avg prompt throughput: 1485.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:18.176599373Z INFO 07-23 02:57:18 metrics.py:345] Avg prompt throughput: 2667.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:24.913022642Z INFO 07-23 02:57:24 metrics.py:345] Avg prompt throughput: 1824.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:32.034111195Z INFO 07-23 02:57:32 metrics.py:345] Avg prompt throughput: 1150.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:42.034654469Z INFO 07-23 02:57:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:47.041274003Z INFO 07-23 02:57:47 metrics.py:345] Avg prompt throughput: 804.5 tokens/s, Avg generation throughput: 7.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:51.606342181Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:57:51.804362711Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:57:53.676250964Z INFO 07-23 02:57:53 metrics.py:345] Avg prompt throughput: 617.3 tokens/s, Avg generation throughput: 13.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:59.365803779Z INFO 07-23 02:57:59 metrics.py:345] Avg prompt throughput: 1703.3 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:04.393685506Z INFO 07-23 02:58:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:04.609577440Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:04.615677229Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:04.798268401Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:58:10.219376350Z INFO 07-23 02:58:10 metrics.py:345] Avg prompt throughput: 2109.3 tokens/s, Avg generation throughput: 0.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:15.231494329Z INFO 07-23 02:58:15 metrics.py:345] Avg prompt throughput: 266.8 tokens/s, Avg generation throughput: 13.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:16.102093564Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:16.116114529Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:16.122546146Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:16.358547897Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:58:21.798309841Z INFO 07-23 02:58:21 metrics.py:345] Avg prompt throughput: 1871.3 tokens/s, Avg generation throughput: 2.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:27.758005289Z INFO 07-23 02:58:27 metrics.py:345] Avg prompt throughput: 2061.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:34.479363188Z INFO 07-23 02:58:34 metrics.py:345] Avg prompt throughput: 1828.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:41.994543666Z INFO 07-23 02:58:41 metrics.py:345] Avg prompt throughput: 1635.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:46.793774434Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:58:48.658622582Z INFO 07-23 02:58:48 metrics.py:345] Avg prompt throughput: 631.3 tokens/s, Avg generation throughput: 11.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:53.685872484Z INFO 07-23 02:58:53 metrics.py:345] Avg prompt throughput: 764.6 tokens/s, Avg generation throughput: 8.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:58.727252608Z INFO 07-23 02:58:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:01.342048508Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:59:01.502531224Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:59:05.905542940Z INFO 07-23 02:59:05 metrics.py:345] Avg prompt throughput: 1007.1 tokens/s, Avg generation throughput: 8.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:10.916217422Z INFO 07-23 02:59:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:15.938840078Z INFO 07-23 02:59:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:16.919636112Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:59:22.003161588Z INFO 07-23 02:59:22 metrics.py:345] Avg prompt throughput: 1326.5 tokens/s, Avg generation throughput: 3.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:27.012505251Z INFO 07-23 02:59:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:31.000804350Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:59:32.036163466Z INFO 07-23 02:59:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 17.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:37.075964266Z INFO 07-23 02:59:37 metrics.py:345] Avg prompt throughput: 1428.0 tokens/s, Avg generation throughput: 7.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:42.093095363Z INFO 07-23 02:59:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:44.048812647Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:59:47.672900564Z INFO 07-23 02:59:47 metrics.py:345] Avg prompt throughput: 1468.2 tokens/s, Avg generation throughput: 6.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:53.382936907Z INFO 07-23 02:59:53 metrics.py:345] Avg prompt throughput: 2152.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:59.847651377Z INFO 07-23 02:59:59 metrics.py:345] Avg prompt throughput: 1900.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:12.037275698Z INFO 07-23 03:00:12 metrics.py:345] Avg prompt throughput: 336.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:17.039468718Z INFO 07-23 03:00:17 metrics.py:345] Avg prompt throughput: 728.1 tokens/s, Avg generation throughput: 12.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:20.892612495Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:00:22.757922531Z INFO 07-23 03:00:22 metrics.py:345] Avg prompt throughput: 716.3 tokens/s, Avg generation throughput: 12.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:27.762148011Z INFO 07-23 03:00:27 metrics.py:345] Avg prompt throughput: 990.8 tokens/s, Avg generation throughput: 9.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:32.765102431Z INFO 07-23 03:00:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:37.783726602Z INFO 07-23 03:00:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:41.716682007Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:00:43.588916994Z INFO 07-23 03:00:43 metrics.py:345] Avg prompt throughput: 705.6 tokens/s, Avg generation throughput: 14.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:48.604975727Z INFO 07-23 03:00:48 metrics.py:345] Avg prompt throughput: 877.8 tokens/s, Avg generation throughput: 12.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:53.636064513Z INFO 07-23 03:00:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:58.643811919Z INFO 07-23 03:00:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:03.437865787Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:01:04.011675373Z INFO 07-23 03:01:04 metrics.py:345] Avg prompt throughput: 134.5 tokens/s, Avg generation throughput: 19.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:09.032151825Z INFO 07-23 03:01:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:14.076641438Z INFO 07-23 03:01:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:19.116791927Z INFO 07-23 03:01:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:24.132816105Z INFO 07-23 03:01:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:29.152510624Z INFO 07-23 03:01:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:34.164408415Z INFO 07-23 03:01:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:39.177405529Z INFO 07-23 03:01:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:44.196112311Z INFO 07-23 03:01:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:49.218177392Z INFO 07-23 03:01:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:54.260843117Z INFO 07-23 03:01:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:59.263581709Z INFO 07-23 03:01:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:04.286227306Z INFO 07-23 03:02:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:09.307925835Z INFO 07-23 03:02:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:14.334136745Z INFO 07-23 03:02:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:19.371566476Z INFO 07-23 03:02:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:24.403118388Z INFO 07-23 03:02:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:29.442979636Z INFO 07-23 03:02:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:34.460345650Z INFO 07-23 03:02:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:34.943043684Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:02:39.543923430Z INFO 07-23 03:02:39 metrics.py:345] Avg prompt throughput: 1440.3 tokens/s, Avg generation throughput: 1.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:43.567655344Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:02:43.733772678Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:02:45.627591234Z INFO 07-23 03:02:45 metrics.py:345] Avg prompt throughput: 673.3 tokens/s, Avg generation throughput: 14.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:50.087651530Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:02:50.253227675Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:02:52.041399788Z INFO 07-23 03:02:52 metrics.py:345] Avg prompt throughput: 556.3 tokens/s, Avg generation throughput: 10.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:57.051443405Z INFO 07-23 03:02:57 metrics.py:345] Avg prompt throughput: 1924.7 tokens/s, Avg generation throughput: 7.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:02.093327354Z INFO 07-23 03:03:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:07.102889862Z INFO 07-23 03:03:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:07.595301247Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:03:12.607251038Z INFO 07-23 03:03:12 metrics.py:345] Avg prompt throughput: 1430.3 tokens/s, Avg generation throughput: 1.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:14.974811119Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:03:20.025072743Z INFO 07-23 03:03:20 metrics.py:345] Avg prompt throughput: 1075.1 tokens/s, Avg generation throughput: 6.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:22.619326919Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:03:26.117782285Z INFO 07-23 03:03:26 metrics.py:345] Avg prompt throughput: 1344.6 tokens/s, Avg generation throughput: 8.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 8.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:31.802449723Z INFO 07-23 03:03:31 metrics.py:345] Avg prompt throughput: 2161.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 8.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:40.400462617Z INFO 07-23 03:03:40 metrics.py:345] Avg prompt throughput: 786.3 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 8.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:40.860411102Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:03:46.589024290Z INFO 07-23 03:03:46 metrics.py:345] Avg prompt throughput: 1610.1 tokens/s, Avg generation throughput: 1.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:51.589997032Z INFO 07-23 03:03:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:56.633889073Z INFO 07-23 03:03:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:57.579527268Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:03:57.746631502Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:04:02.045624284Z INFO 07-23 03:04:02 metrics.py:345] Avg prompt throughput: 756.9 tokens/s, Avg generation throughput: 3.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:04.802148446Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:04:05.027520323Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:04:08.674733439Z INFO 07-23 03:04:08 metrics.py:345] Avg prompt throughput: 1849.3 tokens/s, Avg generation throughput: 6.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:14.390439518Z INFO 07-23 03:04:14 metrics.py:345] Avg prompt throughput: 2149.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:20.860969569Z INFO 07-23 03:04:20 metrics.py:345] Avg prompt throughput: 1899.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:28.091096635Z INFO 07-23 03:04:28 metrics.py:345] Avg prompt throughput: 1699.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:33.666207138Z INFO 07-23 03:04:33 metrics.py:345] Avg prompt throughput: 832.6 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:37.651979949Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:04:38.689261053Z INFO 07-23 03:04:38 metrics.py:345] Avg prompt throughput: 108.1 tokens/s, Avg generation throughput: 17.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:43.719316648Z INFO 07-23 03:04:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:48.726194909Z INFO 07-23 03:04:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:53.773772705Z INFO 07-23 03:04:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:58.783851267Z INFO 07-23 03:04:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:03.796039763Z INFO 07-23 03:05:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:08.800370314Z INFO 07-23 03:05:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:13.817925374Z INFO 07-23 03:05:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:18.827534721Z INFO 07-23 03:05:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:23.836345906Z INFO 07-23 03:05:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:28.867665562Z INFO 07-23 03:05:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:33.879535202Z INFO 07-23 03:05:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:38.887930697Z INFO 07-23 03:05:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:43.911415061Z INFO 07-23 03:05:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:48.919604312Z INFO 07-23 03:05:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:53.923735935Z INFO 07-23 03:05:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:58.937847984Z INFO 07-23 03:05:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:03.976626873Z INFO 07-23 03:06:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:08.988814054Z INFO 07-23 03:06:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:13.996690644Z INFO 07-23 03:06:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:19.022685216Z INFO 07-23 03:06:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:24.035035366Z INFO 07-23 03:06:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:29.045998728Z INFO 07-23 03:06:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:34.065591058Z INFO 07-23 03:06:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:39.065937087Z INFO 07-23 03:06:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:44.089850754Z INFO 07-23 03:06:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:49.122224783Z INFO 07-23 03:06:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:54.140258854Z INFO 07-23 03:06:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:59.155775804Z INFO 07-23 03:06:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:04.167641219Z INFO 07-23 03:07:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:09.183289842Z INFO 07-23 03:07:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:14.190459782Z INFO 07-23 03:07:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:16.063329774Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:07:19.693310580Z INFO 07-23 03:07:19 metrics.py:345] Avg prompt throughput: 1488.7 tokens/s, Avg generation throughput: 6.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:24.709589906Z INFO 07-23 03:07:24 metrics.py:345] Avg prompt throughput: 289.5 tokens/s, Avg generation throughput: 15.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:29.752714538Z INFO 07-23 03:07:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:34.754784357Z INFO 07-23 03:07:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:37.046291229Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:07:40.663717101Z INFO 07-23 03:07:40 metrics.py:345] Avg prompt throughput: 1386.4 tokens/s, Avg generation throughput: 8.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:45.698396136Z INFO 07-23 03:07:45 metrics.py:345] Avg prompt throughput: 1079.9 tokens/s, Avg generation throughput: 5.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:50.739722502Z INFO 07-23 03:07:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:55.759442021Z INFO 07-23 03:07:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:59.292075461Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:08:00.760756817Z INFO 07-23 03:08:00 metrics.py:345] Avg prompt throughput: 68.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:05.803486926Z INFO 07-23 03:08:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:10.807800210Z INFO 07-23 03:08:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:15.810120875Z INFO 07-23 03:08:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:20.821109568Z INFO 07-23 03:08:20 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:25.834563489Z INFO 07-23 03:08:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:30.852701551Z INFO 07-23 03:08:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:35.895279345Z INFO 07-23 03:08:35 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:40.898125953Z INFO 07-23 03:08:40 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:45.903238274Z INFO 07-23 03:08:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:50.911176380Z INFO 07-23 03:08:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:55.948442038Z INFO 07-23 03:08:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:00.986020640Z INFO 07-23 03:09:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:05.987833861Z INFO 07-23 03:09:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:10.999189231Z INFO 07-23 03:09:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:16.010005913Z INFO 07-23 03:09:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:21.023415174Z INFO 07-23 03:09:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:26.037961024Z INFO 07-23 03:09:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:31.059518782Z INFO 07-23 03:09:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:36.076762327Z INFO 07-23 03:09:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:41.098050215Z INFO 07-23 03:09:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:46.109931334Z INFO 07-23 03:09:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:51.117802613Z INFO 07-23 03:09:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:52.198541671Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:09:58.473656203Z INFO 07-23 03:09:58 metrics.py:345] Avg prompt throughput: 1424.5 tokens/s, Avg generation throughput: 2.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:03.483554227Z INFO 07-23 03:10:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:08.486025664Z INFO 07-23 03:10:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:13.387287348Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:10:13.573126997Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:10:15.451842926Z INFO 07-23 03:10:15 metrics.py:345] Avg prompt throughput: 588.0 tokens/s, Avg generation throughput: 15.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:20.484742816Z INFO 07-23 03:10:20 metrics.py:345] Avg prompt throughput: 1315.2 tokens/s, Avg generation throughput: 2.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:25.490834189Z INFO 07-23 03:10:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:30.496012765Z INFO 07-23 03:10:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:33.890420869Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:10:35.776362721Z INFO 07-23 03:10:35 metrics.py:345] Avg prompt throughput: 775.7 tokens/s, Avg generation throughput: 13.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:40.799476081Z INFO 07-23 03:10:40 metrics.py:345] Avg prompt throughput: 834.3 tokens/s, Avg generation throughput: 13.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:42.302078280Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:10:45.904718727Z INFO 07-23 03:10:45 metrics.py:345] Avg prompt throughput: 1604.6 tokens/s, Avg generation throughput: 5.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:50.918196398Z INFO 07-23 03:10:50 metrics.py:345] Avg prompt throughput: 43.5 tokens/s, Avg generation throughput: 20.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:51.861459428Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:10:51.868181568Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:10:52.095460779Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:10:57.543735685Z INFO 07-23 03:10:57 metrics.py:345] Avg prompt throughput: 1854.6 tokens/s, Avg generation throughput: 3.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:03.506535562Z INFO 07-23 03:11:03 metrics.py:345] Avg prompt throughput: 2060.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:10.241413771Z INFO 07-23 03:11:10 metrics.py:345] Avg prompt throughput: 1824.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:17.765981052Z INFO 07-23 03:11:17 metrics.py:345] Avg prompt throughput: 1633.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:22.772423832Z INFO 07-23 03:11:22 metrics.py:345] Avg prompt throughput: 173.2 tokens/s, Avg generation throughput: 3.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:27.820717941Z INFO 07-23 03:11:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:32.868761969Z INFO 07-23 03:11:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:35.822104096Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:11:39.432629061Z INFO 07-23 03:11:39 metrics.py:345] Avg prompt throughput: 1248.1 tokens/s, Avg generation throughput: 8.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:42.539380381Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:11:46.145813361Z INFO 07-23 03:11:46 metrics.py:345] Avg prompt throughput: 1271.2 tokens/s, Avg generation throughput: 7.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:50.849891660Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:11:50.859074820Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:11:50.870425879Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:11:51.098867253Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:11:52.776745737Z INFO 07-23 03:11:52 metrics.py:345] Avg prompt throughput: 71.2 tokens/s, Avg generation throughput: 12.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:57.791435469Z INFO 07-23 03:11:57 metrics.py:345] Avg prompt throughput: 1751.5 tokens/s, Avg generation throughput: 10.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:58.518847773Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:11:58.688843761Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:12:04.113810544Z INFO 07-23 03:12:04 metrics.py:345] Avg prompt throughput: 1943.6 tokens/s, Avg generation throughput: 2.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:09.124640595Z INFO 07-23 03:12:09 metrics.py:345] Avg prompt throughput: 450.4 tokens/s, Avg generation throughput: 7.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:14.096840115Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:12:14.320275730Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:12:16.080406340Z INFO 07-23 03:12:16 metrics.py:345] Avg prompt throughput: 588.9 tokens/s, Avg generation throughput: 16.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:21.516014892Z INFO 07-23 03:12:21 metrics.py:345] Avg prompt throughput: 2260.7 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:32.515215161Z INFO 07-23 03:12:32 metrics.py:345] Avg prompt throughput: 1016.9 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:33.019477144Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:12:41.016003493Z INFO 07-23 03:12:41 metrics.py:345] Avg prompt throughput: 1415.5 tokens/s, Avg generation throughput: 0.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:46.059272778Z INFO 07-23 03:12:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:51.061081047Z INFO 07-23 03:12:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:56.072726396Z INFO 07-23 03:12:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:58.413592732Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:13:02.114424022Z INFO 07-23 03:13:02 metrics.py:345] Avg prompt throughput: 1355.9 tokens/s, Avg generation throughput: 7.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:07.138412366Z INFO 07-23 03:13:07 metrics.py:345] Avg prompt throughput: 1180.3 tokens/s, Avg generation throughput: 2.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:10.703528957Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:10.723774197Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:10.959621972Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:13:12.777150696Z INFO 07-23 03:13:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 14.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:18.311467651Z INFO 07-23 03:13:18 metrics.py:345] Avg prompt throughput: 2960.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:24.528764448Z INFO 07-23 03:13:24 metrics.py:345] Avg prompt throughput: 1976.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:31.514201171Z INFO 07-23 03:13:31 metrics.py:345] Avg prompt throughput: 1759.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:42.028574197Z INFO 07-23 03:13:42 metrics.py:345] Avg prompt throughput: 277.9 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:45.431039577Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:45.437698822Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:45.444970744Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:45.625709408Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:13:47.511514890Z INFO 07-23 03:13:47 metrics.py:345] Avg prompt throughput: 747.0 tokens/s, Avg generation throughput: 12.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:52.776792073Z INFO 07-23 03:13:52 metrics.py:345] Avg prompt throughput: 777.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:57.800309946Z INFO 07-23 03:13:57 metrics.py:345] Avg prompt throughput: 690.0 tokens/s, Avg generation throughput: 21.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:02.809048544Z INFO 07-23 03:14:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:06.461208755Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:14:06.468193202Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:14:06.479204061Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:14:06.490777942Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:14:06.648810696Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:14:07.835638500Z INFO 07-23 03:14:07 metrics.py:345] Avg prompt throughput: 24.3 tokens/s, Avg generation throughput: 20.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:12.846371069Z INFO 07-23 03:14:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:17.850632021Z INFO 07-23 03:14:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:22.856280984Z INFO 07-23 03:14:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:27.888165813Z INFO 07-23 03:14:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:32.925170980Z INFO 07-23 03:14:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:37.936372523Z INFO 07-23 03:14:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:42.944534615Z INFO 07-23 03:14:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:47.945186825Z INFO 07-23 03:14:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:52.949910632Z INFO 07-23 03:14:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:57.964030722Z INFO 07-23 03:14:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:02.982507987Z INFO 07-23 03:15:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:07.998544641Z INFO 07-23 03:15:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:13.004333260Z INFO 07-23 03:15:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:18.016318554Z INFO 07-23 03:15:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:23.029382219Z INFO 07-23 03:15:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:28.041986622Z INFO 07-23 03:15:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:33.065708355Z INFO 07-23 03:15:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:38.078323753Z INFO 07-23 03:15:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:43.091103774Z INFO 07-23 03:15:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:48.108899253Z INFO 07-23 03:15:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:53.135039995Z INFO 07-23 03:15:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:58.165410734Z INFO 07-23 03:15:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:03.183514389Z INFO 07-23 03:16:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:08.205135732Z INFO 07-23 03:16:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:13.216887456Z INFO 07-23 03:16:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:18.224956585Z INFO 07-23 03:16:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:20.572927902Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:16:20.746589986Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:16:23.267047992Z INFO 07-23 03:16:23 metrics.py:345] Avg prompt throughput: 107.7 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:28.282762875Z INFO 07-23 03:16:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:33.296712243Z INFO 07-23 03:16:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:38.326019630Z INFO 07-23 03:16:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:43.357754792Z INFO 07-23 03:16:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:48.372927578Z INFO 07-23 03:16:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:53.390454474Z INFO 07-23 03:16:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:58.399988303Z INFO 07-23 03:16:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:03.405932485Z INFO 07-23 03:17:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:08.446484958Z INFO 07-23 03:17:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:13.485751050Z INFO 07-23 03:17:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:18.523721424Z INFO 07-23 03:17:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:23.560268144Z INFO 07-23 03:17:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:28.594709480Z INFO 07-23 03:17:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:33.630996999Z INFO 07-23 03:17:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:38.670563497Z INFO 07-23 03:17:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:43.672950199Z INFO 07-23 03:17:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:48.687444828Z INFO 07-23 03:17:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:53.687871665Z INFO 07-23 03:17:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:58.699681518Z INFO 07-23 03:17:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:03.710114472Z INFO 07-23 03:18:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:08.720217881Z INFO 07-23 03:18:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:13.743844160Z INFO 07-23 03:18:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:18.764123061Z INFO 07-23 03:18:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:23.778691605Z INFO 07-23 03:18:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:28.790043878Z INFO 07-23 03:18:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:33.801253262Z INFO 07-23 03:18:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:38.807935708Z INFO 07-23 03:18:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:43.815867459Z INFO 07-23 03:18:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:48.827006448Z INFO 07-23 03:18:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:53.834181643Z INFO 07-23 03:18:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:58.865607506Z INFO 07-23 03:18:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:03.879818981Z INFO 07-23 03:19:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:08.900000692Z INFO 07-23 03:19:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:10.466463776Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:19:10.478993493Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:19:10.679321643Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:19:14.309575673Z INFO 07-23 03:19:14 metrics.py:345] Avg prompt throughput: 1514.4 tokens/s, Avg generation throughput: 6.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:19.322028134Z INFO 07-23 03:19:19 metrics.py:345] Avg prompt throughput: 972.8 tokens/s, Avg generation throughput: 8.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:24.355050263Z INFO 07-23 03:19:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:27.145073359Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:19:31.802504328Z INFO 07-23 03:19:31 metrics.py:345] Avg prompt throughput: 1008.4 tokens/s, Avg generation throughput: 8.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:36.308014528Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:19:36.316129768Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:19:36.507397695Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:19:38.394733836Z INFO 07-23 03:19:38 metrics.py:345] Avg prompt throughput: 621.3 tokens/s, Avg generation throughput: 15.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:43.661430418Z INFO 07-23 03:19:43 metrics.py:345] Avg prompt throughput: 1761.8 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:48.702576396Z INFO 07-23 03:19:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:53.735280536Z INFO 07-23 03:19:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:55.502354844Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:20:00.322746405Z INFO 07-23 03:20:00 metrics.py:345] Avg prompt throughput: 1168.0 tokens/s, Avg generation throughput: 5.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:02.888919108Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:20:02.897148113Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:20:03.074751698Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:20:06.698577320Z INFO 07-23 03:20:06 metrics.py:345] Avg prompt throughput: 1284.8 tokens/s, Avg generation throughput: 9.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:11.953751610Z INFO 07-23 03:20:11 metrics.py:345] Avg prompt throughput: 1225.3 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:16.958423659Z INFO 07-23 03:20:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:21.958724991Z INFO 07-23 03:20:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:26.962080264Z INFO 07-23 03:20:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:31.188353870Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:20:31.346968400Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:20:31.986096383Z INFO 07-23 03:20:31 metrics.py:345] Avg prompt throughput: 65.5 tokens/s, Avg generation throughput: 19.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:36.991956897Z INFO 07-23 03:20:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:42.022761244Z INFO 07-23 03:20:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:47.065410115Z INFO 07-23 03:20:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:52.071533963Z INFO 07-23 03:20:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:57.095045358Z INFO 07-23 03:20:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:02.100354460Z INFO 07-23 03:21:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:07.120462554Z INFO 07-23 03:21:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:12.144278143Z INFO 07-23 03:21:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:17.160746225Z INFO 07-23 03:21:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:22.183849444Z INFO 07-23 03:21:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:27.201981662Z INFO 07-23 03:21:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:32.222207283Z INFO 07-23 03:21:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:37.226790110Z INFO 07-23 03:21:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:42.263162787Z INFO 07-23 03:21:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:47.285217965Z INFO 07-23 03:21:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:52.294860803Z INFO 07-23 03:21:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:57.305607847Z INFO 07-23 03:21:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:02.332677933Z INFO 07-23 03:22:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:07.361450993Z INFO 07-23 03:22:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:12.386346091Z INFO 07-23 03:22:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:17.425448234Z INFO 07-23 03:22:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:22.445119521Z INFO 07-23 03:22:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:27.467709607Z INFO 07-23 03:22:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:32.488520263Z INFO 07-23 03:22:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:37.510941058Z INFO 07-23 03:22:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:42.548234171Z INFO 07-23 03:22:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:47.581833368Z INFO 07-23 03:22:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:52.623687606Z INFO 07-23 03:22:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:57.640445190Z INFO 07-23 03:22:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:02.662738298Z INFO 07-23 03:23:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:04.568763829Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:23:04.740545928Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:23:07.680005320Z INFO 07-23 03:23:07 metrics.py:345] Avg prompt throughput: 228.6 tokens/s, Avg generation throughput: 18.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:12.697581987Z INFO 07-23 03:23:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:17.717154268Z INFO 07-23 03:23:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:22.737304950Z INFO 07-23 03:23:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:27.757062716Z INFO 07-23 03:23:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:32.761707602Z INFO 07-23 03:23:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:37.779115679Z INFO 07-23 03:23:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:42.787757479Z INFO 07-23 03:23:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:47.821253928Z INFO 07-23 03:23:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:52.843645340Z INFO 07-23 03:23:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:57.888462280Z INFO 07-23 03:23:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:02.888897990Z INFO 07-23 03:24:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:07.905867858Z INFO 07-23 03:24:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:12.928719262Z INFO 07-23 03:24:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:17.947424275Z INFO 07-23 03:24:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:22.973047988Z INFO 07-23 03:24:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:27.993404362Z INFO 07-23 03:24:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:33.038567398Z INFO 07-23 03:24:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:38.072799979Z INFO 07-23 03:24:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:43.107134360Z INFO 07-23 03:24:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:48.139340196Z INFO 07-23 03:24:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:53.158864324Z INFO 07-23 03:24:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:58.184946563Z INFO 07-23 03:24:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:03.205227792Z INFO 07-23 03:25:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:08.227341986Z INFO 07-23 03:25:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:13.266499819Z INFO 07-23 03:25:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:18.273249978Z INFO 07-23 03:25:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:23.310387392Z INFO 07-23 03:25:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:28.347385529Z INFO 07-23 03:25:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:33.372635744Z INFO 07-23 03:25:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:38.389145564Z INFO 07-23 03:25:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:43.404210116Z INFO 07-23 03:25:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:48.431958800Z INFO 07-23 03:25:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:53.458237948Z INFO 07-23 03:25:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:58.483877167Z INFO 07-23 03:25:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:03.522336181Z INFO 07-23 03:26:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:05.090021844Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:05.095951361Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:05.295502128Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:26:08.926311950Z INFO 07-23 03:26:08 metrics.py:345] Avg prompt throughput: 1516.0 tokens/s, Avg generation throughput: 6.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:14.842702111Z INFO 07-23 03:26:14 metrics.py:345] Avg prompt throughput: 1164.9 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:19.852487057Z INFO 07-23 03:26:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:20.202580536Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:20.210313004Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:20.384973960Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:26:24.999260358Z INFO 07-23 03:26:24 metrics.py:345] Avg prompt throughput: 1444.2 tokens/s, Avg generation throughput: 1.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:30.041607883Z INFO 07-23 03:26:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:35.048754058Z INFO 07-23 03:26:35 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:39.100631558Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:39.109347200Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:39.304823096Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:26:41.188117195Z INFO 07-23 03:26:41 metrics.py:345] Avg prompt throughput: 667.2 tokens/s, Avg generation throughput: 15.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:50.433459245Z INFO 07-23 03:26:50 metrics.py:345] Avg prompt throughput: 1319.3 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:55.457907052Z INFO 07-23 03:26:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:00.476478203Z INFO 07-23 03:27:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:01.588214832Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:27:07.025171015Z INFO 07-23 03:27:07 metrics.py:345] Avg prompt throughput: 1876.4 tokens/s, Avg generation throughput: 3.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:12.629720324Z INFO 07-23 03:27:12 metrics.py:345] Avg prompt throughput: 1105.2 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:17.668027044Z INFO 07-23 03:27:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:22.673085829Z INFO 07-23 03:27:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:27.691946553Z INFO 07-23 03:27:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:29.151414399Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:27:29.743848827Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:27:29.763701013Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:27:29.917975218Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:27:32.793393663Z INFO 07-23 03:27:32 metrics.py:345] Avg prompt throughput: 802.9 tokens/s, Avg generation throughput: 6.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:37.836214318Z INFO 07-23 03:27:37 metrics.py:345] Avg prompt throughput: 708.7 tokens/s, Avg generation throughput: 14.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:42.859098255Z INFO 07-23 03:27:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:44.256741458Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:27:47.875184669Z INFO 07-23 03:27:47 metrics.py:345] Avg prompt throughput: 1633.2 tokens/s, Avg generation throughput: 5.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:52.899691834Z INFO 07-23 03:27:52 metrics.py:345] Avg prompt throughput: 155.6 tokens/s, Avg generation throughput: 18.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:57.919289532Z INFO 07-23 03:27:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:02.702062753Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:28:02.873611630Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:28:03.112144515Z INFO 07-23 03:28:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.6 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:08.150500129Z INFO 07-23 03:28:08 metrics.py:345] Avg prompt throughput: 1578.7 tokens/s, Avg generation throughput: 1.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:13.187908126Z INFO 07-23 03:28:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:17.164320870Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:28:17.327688086Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:28:19.210550904Z INFO 07-23 03:28:19 metrics.py:345] Avg prompt throughput: 680.1 tokens/s, Avg generation throughput: 14.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:24.225292173Z INFO 07-23 03:28:24 metrics.py:345] Avg prompt throughput: 800.0 tokens/s, Avg generation throughput: 8.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:29.226764995Z INFO 07-23 03:28:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:31.869759250Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:28:34.263598007Z INFO 07-23 03:28:34 metrics.py:345] Avg prompt throughput: 132.0 tokens/s, Avg generation throughput: 18.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:39.292111567Z INFO 07-23 03:28:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:44.324452648Z INFO 07-23 03:28:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:49.348232646Z INFO 07-23 03:28:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:54.355931285Z INFO 07-23 03:28:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:59.363795090Z INFO 07-23 03:28:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:04.392516862Z INFO 07-23 03:29:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:09.415443309Z INFO 07-23 03:29:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:14.436739524Z INFO 07-23 03:29:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:19.457395216Z INFO 07-23 03:29:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:24.474718167Z INFO 07-23 03:29:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:29.501523520Z INFO 07-23 03:29:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:34.523233969Z INFO 07-23 03:29:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:39.552660459Z INFO 07-23 03:29:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:44.570080552Z INFO 07-23 03:29:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:49.596454253Z INFO 07-23 03:29:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:54.615672916Z INFO 07-23 03:29:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:59.633932098Z INFO 07-23 03:29:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:02.589700120Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:02.778803816Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:04.677928907Z INFO 07-23 03:30:04 metrics.py:345] Avg prompt throughput: 812.1 tokens/s, Avg generation throughput: 13.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:09.720978113Z INFO 07-23 03:30:09 metrics.py:345] Avg prompt throughput: 952.0 tokens/s, Avg generation throughput: 10.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:11.253121488Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:14.873610618Z INFO 07-23 03:30:14 metrics.py:345] Avg prompt throughput: 1589.9 tokens/s, Avg generation throughput: 5.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:21.905152531Z INFO 07-23 03:30:21 metrics.py:345] Avg prompt throughput: 1420.5 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:26.927246350Z INFO 07-23 03:30:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:26.949212854Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:26.957860631Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:26.966418697Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:27.141252545Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:31.939042290Z INFO 07-23 03:30:31 metrics.py:345] Avg prompt throughput: 1712.4 tokens/s, Avg generation throughput: 2.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:36.949470944Z INFO 07-23 03:30:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:40.533602444Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:40.691497271Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:41.010347463Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:41.018689226Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:41.178014687Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:43.067206037Z INFO 07-23 03:30:43 metrics.py:345] Avg prompt throughput: 685.9 tokens/s, Avg generation throughput: 13.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:48.085902193Z INFO 07-23 03:30:48 metrics.py:345] Avg prompt throughput: 617.9 tokens/s, Avg generation throughput: 11.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:53.113753547Z INFO 07-23 03:30:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:53.864374472Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:53.875008951Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:54.049288150Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:58.801164893Z INFO 07-23 03:30:58 metrics.py:345] Avg prompt throughput: 1346.1 tokens/s, Avg generation throughput: 3.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:01.135308567Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:31:06.029877137Z INFO 07-23 03:31:06 metrics.py:345] Avg prompt throughput: 1074.5 tokens/s, Avg generation throughput: 6.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:08.592483770Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:31:08.960823866Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:31:12.640101178Z INFO 07-23 03:31:12 metrics.py:345] Avg prompt throughput: 1239.3 tokens/s, Avg generation throughput: 8.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:18.383940479Z INFO 07-23 03:31:18 metrics.py:345] Avg prompt throughput: 2139.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:24.879158343Z INFO 07-23 03:31:24 metrics.py:345] Avg prompt throughput: 1891.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:32.155291335Z INFO 07-23 03:31:32 metrics.py:345] Avg prompt throughput: 1688.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:37.443808749Z INFO 07-23 03:31:37 metrics.py:345] Avg prompt throughput: 1549.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:43.111724266Z INFO 07-23 03:31:43 metrics.py:345] Avg prompt throughput: 1445.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:49.164283827Z INFO 07-23 03:31:49 metrics.py:345] Avg prompt throughput: 1353.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:55.618757782Z INFO 07-23 03:31:55 metrics.py:345] Avg prompt throughput: 1269.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:02.450252476Z INFO 07-23 03:32:02 metrics.py:345] Avg prompt throughput: 1199.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:09.683638726Z INFO 07-23 03:32:09 metrics.py:345] Avg prompt throughput: 1132.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:17.325293296Z INFO 07-23 03:32:17 metrics.py:345] Avg prompt throughput: 1072.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:23.119540943Z INFO 07-23 03:32:23 metrics.py:345] Avg prompt throughput: 706.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:33.119590916Z INFO 07-23 03:32:33 metrics.py:345] Avg prompt throughput: 409.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:43.120993751Z INFO 07-23 03:32:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:51.139668086Z INFO 07-23 03:32:51 metrics.py:345] Avg prompt throughput: 349.9 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:56.182102278Z INFO 07-23 03:32:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:01.222431844Z INFO 07-23 03:33:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:06.255444585Z INFO 07-23 03:33:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:11.294163817Z INFO 07-23 03:33:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:16.330648490Z INFO 07-23 03:33:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:21.366805857Z INFO 07-23 03:33:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:26.401669203Z INFO 07-23 03:33:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:31.427649231Z INFO 07-23 03:33:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:36.462316305Z INFO 07-23 03:33:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:41.488932088Z INFO 07-23 03:33:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:46.518394807Z INFO 07-23 03:33:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:51.547857990Z INFO 07-23 03:33:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:56.580255579Z INFO 07-23 03:33:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:01.580987774Z INFO 07-23 03:34:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:06.616073565Z INFO 07-23 03:34:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:11.650597177Z INFO 07-23 03:34:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:16.654305379Z INFO 07-23 03:34:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:21.683189679Z INFO 07-23 03:34:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:26.707783194Z INFO 07-23 03:34:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:31.739851567Z INFO 07-23 03:34:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:36.764364675Z INFO 07-23 03:34:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:41.785282509Z INFO 07-23 03:34:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:46.807385509Z INFO 07-23 03:34:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:51.832113797Z INFO 07-23 03:34:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:56.852758637Z INFO 07-23 03:34:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:01.877195030Z INFO 07-23 03:35:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:06.909124996Z INFO 07-23 03:35:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:11.936863441Z INFO 07-23 03:35:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:16.969294745Z INFO 07-23 03:35:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:22.001685150Z INFO 07-23 03:35:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:27.035357579Z INFO 07-23 03:35:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:32.070780441Z INFO 07-23 03:35:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:37.109800889Z INFO 07-23 03:35:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:42.144817317Z INFO 07-23 03:35:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:47.182019980Z INFO 07-23 03:35:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:52.225007586Z INFO 07-23 03:35:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:57.259046506Z INFO 07-23 03:35:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:02.297575559Z INFO 07-23 03:36:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:07.340493043Z INFO 07-23 03:36:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:12.378520548Z INFO 07-23 03:36:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:17.409447662Z INFO 07-23 03:36:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:22.439119517Z INFO 07-23 03:36:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:27.442704506Z INFO 07-23 03:36:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:32.480948868Z INFO 07-23 03:36:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:37.529398347Z INFO 07-23 03:36:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:42.574099184Z INFO 07-23 03:36:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:47.612483559Z INFO 07-23 03:36:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:52.652951855Z INFO 07-23 03:36:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:57.687904692Z INFO 07-23 03:36:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:02.726593377Z INFO 07-23 03:37:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:07.730913728Z INFO 07-23 03:37:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:12.774245347Z INFO 07-23 03:37:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:17.813872698Z INFO 07-23 03:37:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:22.848066921Z INFO 07-23 03:37:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:27.882473147Z INFO 07-23 03:37:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:32.917401272Z INFO 07-23 03:37:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:37.932243776Z INFO 07-23 03:37:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:42.963779527Z INFO 07-23 03:37:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:48.007317544Z INFO 07-23 03:37:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:53.048938883Z INFO 07-23 03:37:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:58.085090714Z INFO 07-23 03:37:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:59.337134045Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:03.126573524Z INFO 07-23 03:38:03 metrics.py:345] Avg prompt throughput: 1624.9 tokens/s, Avg generation throughput: 4.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:07.915203230Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:38:07.925057068Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:38:08.094963074Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:09.982564809Z INFO 07-23 03:38:09 metrics.py:345] Avg prompt throughput: 715.7 tokens/s, Avg generation throughput: 12.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:14.987905774Z INFO 07-23 03:38:14 metrics.py:345] Avg prompt throughput: 1116.8 tokens/s, Avg generation throughput: 7.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:19.993703211Z INFO 07-23 03:38:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:23.734983446Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:25.623769661Z INFO 07-23 03:38:25 metrics.py:345] Avg prompt throughput: 727.5 tokens/s, Avg generation throughput: 14.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:30.635619226Z INFO 07-23 03:38:30 metrics.py:345] Avg prompt throughput: 1021.6 tokens/s, Avg generation throughput: 12.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:34.343758361Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:38:34.566766277Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:36.464600680Z INFO 07-23 03:38:36 metrics.py:345] Avg prompt throughput: 702.7 tokens/s, Avg generation throughput: 14.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:41.913518197Z INFO 07-23 03:38:41 metrics.py:345] Avg prompt throughput: 2255.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:53.127802320Z INFO 07-23 03:38:53 metrics.py:345] Avg prompt throughput: 730.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:57.626063560Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:59.523823531Z INFO 07-23 03:38:59 metrics.py:345] Avg prompt throughput: 1128.4 tokens/s, Avg generation throughput: 14.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:04.973947484Z INFO 07-23 03:39:04 metrics.py:345] Avg prompt throughput: 2254.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:13.127817976Z INFO 07-23 03:39:13 metrics.py:345] Avg prompt throughput: 502.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:18.136657170Z INFO 07-23 03:39:18 metrics.py:345] Avg prompt throughput: 697.6 tokens/s, Avg generation throughput: 16.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:19.550416132Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:39:23.976378415Z INFO 07-23 03:39:23 metrics.py:345] Avg prompt throughput: 1237.9 tokens/s, Avg generation throughput: 4.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:29.002950812Z INFO 07-23 03:39:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:34.041860533Z INFO 07-23 03:39:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:34.620637687Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:39:41.578722845Z INFO 07-23 03:39:41 metrics.py:345] Avg prompt throughput: 1490.8 tokens/s, Avg generation throughput: 1.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:46.612817940Z INFO 07-23 03:39:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:51.651629828Z INFO 07-23 03:39:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:56.654212752Z INFO 07-23 03:39:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:58.976469155Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:40:02.603723546Z INFO 07-23 03:40:02 metrics.py:345] Avg prompt throughput: 1376.9 tokens/s, Avg generation throughput: 7.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:08.305651965Z INFO 07-23 03:40:08 metrics.py:345] Avg prompt throughput: 2155.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:16.236480761Z INFO 07-23 03:40:16 metrics.py:345] Avg prompt throughput: 478.5 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:21.267816676Z INFO 07-23 03:40:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:26.295602231Z INFO 07-23 03:40:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 8.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:28.778018078Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:40:32.408324583Z INFO 07-23 03:40:32 metrics.py:345] Avg prompt throughput: 1340.2 tokens/s, Avg generation throughput: 8.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:38.108997575Z INFO 07-23 03:40:38 metrics.py:345] Avg prompt throughput: 2155.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:43.129334599Z INFO 07-23 03:40:43 metrics.py:345] Avg prompt throughput: 1631.7 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:48.160479129Z INFO 07-23 03:40:48 metrics.py:345] Avg prompt throughput: 120.6 tokens/s, Avg generation throughput: 16.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:53.164874777Z INFO 07-23 03:40:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:58.169636135Z INFO 07-23 03:40:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:59.795739240Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:41:03.414206853Z INFO 07-23 03:41:03 metrics.py:345] Avg prompt throughput: 1562.0 tokens/s, Avg generation throughput: 5.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:08.455624617Z INFO 07-23 03:41:08 metrics.py:345] Avg prompt throughput: 728.7 tokens/s, Avg generation throughput: 4.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:13.456923903Z INFO 07-23 03:41:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:18.485525514Z INFO 07-23 03:41:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:23.498236924Z INFO 07-23 03:41:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:26.268303762Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:41:31.070539610Z INFO 07-23 03:41:31 metrics.py:345] Avg prompt throughput: 1015.4 tokens/s, Avg generation throughput: 7.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:36.096941176Z INFO 07-23 03:41:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:41.130726930Z INFO 07-23 03:41:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:41.559287741Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:41:41.969406342Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:41:47.474648078Z INFO 07-23 03:41:47 metrics.py:345] Avg prompt throughput: 1937.0 tokens/s, Avg generation throughput: 1.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:53.131561420Z INFO 07-23 03:41:53 metrics.py:345] Avg prompt throughput: 1448.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:00.176943964Z INFO 07-23 03:42:00 metrics.py:345] Avg prompt throughput: 2325.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:07.700147555Z INFO 07-23 03:42:07 metrics.py:345] Avg prompt throughput: 1633.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:13.132480415Z INFO 07-23 03:42:13 metrics.py:345] Avg prompt throughput: 754.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:19.035572074Z INFO 07-23 03:42:19 metrics.py:345] Avg prompt throughput: 2081.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:33.133304474Z INFO 07-23 03:42:33 metrics.py:345] Avg prompt throughput: 290.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:43.134438022Z INFO 07-23 03:42:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:48.154183452Z INFO 07-23 03:42:48 metrics.py:345] Avg prompt throughput: 735.3 tokens/s, Avg generation throughput: 17.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:53.191208919Z INFO 07-23 03:42:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:57.644819022Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:42:59.536790168Z INFO 07-23 03:42:59 metrics.py:345] Avg prompt throughput: 645.5 tokens/s, Avg generation throughput: 13.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:04.984151481Z INFO 07-23 03:43:04 metrics.py:345] Avg prompt throughput: 2255.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:13.134871432Z INFO 07-23 03:43:13 metrics.py:345] Avg prompt throughput: 1005.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:18.371162432Z INFO 07-23 03:43:18 metrics.py:345] Avg prompt throughput: 757.4 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:23.343108547Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:43:23.588052243Z INFO 07-23 03:43:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:28.848071558Z INFO 07-23 03:43:28 metrics.py:345] Avg prompt throughput: 2336.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:34.822293294Z INFO 07-23 03:43:34 metrics.py:345] Avg prompt throughput: 2056.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:41.559053064Z INFO 07-23 03:43:41 metrics.py:345] Avg prompt throughput: 1824.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:42.134554199Z INFO:     172.28.22.23:54024 - "GET /v1/models HTTP/1.1" 200 OK
+2026-07-23T03:43:49.093692247Z INFO 07-23 03:43:49 metrics.py:345] Avg prompt throughput: 1630.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:54.573982630Z INFO 07-23 03:43:54 metrics.py:345] Avg prompt throughput: 1494.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:00.419430557Z INFO 07-23 03:44:00 metrics.py:345] Avg prompt throughput: 1401.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:06.652667894Z INFO 07-23 03:44:06 metrics.py:345] Avg prompt throughput: 1314.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:13.273466906Z INFO 07-23 03:44:13 metrics.py:345] Avg prompt throughput: 1237.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:23.589944720Z INFO 07-23 03:44:23 metrics.py:345] Avg prompt throughput: 397.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:33.589622516Z INFO 07-23 03:44:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:40.373070879Z INFO 07-23 03:44:40 metrics.py:345] Avg prompt throughput: 477.0 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:45.378779262Z INFO 07-23 03:44:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:49.409870456Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:44:51.294134680Z INFO 07-23 03:44:51 metrics.py:345] Avg prompt throughput: 692.4 tokens/s, Avg generation throughput: 13.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:57.327566458Z INFO 07-23 03:44:57 metrics.py:345] Avg prompt throughput: 1304.6 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:02.343004242Z INFO 07-23 03:45:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:07.353037375Z INFO 07-23 03:45:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:08.703038876Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:45:08.945392988Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:45:12.589865901Z INFO 07-23 03:45:12 metrics.py:345] Avg prompt throughput: 1564.3 tokens/s, Avg generation throughput: 5.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:18.308064429Z INFO 07-23 03:45:18 metrics.py:345] Avg prompt throughput: 2148.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:23.590532919Z INFO 07-23 03:45:23 metrics.py:345] Avg prompt throughput: 1550.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:29.508225758Z INFO 07-23 03:45:29 metrics.py:345] Avg prompt throughput: 2076.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:40.164335600Z INFO 07-23 03:45:40 metrics.py:345] Avg prompt throughput: 286.5 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:45.215691270Z INFO 07-23 03:45:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:50.262555523Z INFO 07-23 03:45:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:55.305684193Z INFO 07-23 03:45:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:00.329990814Z INFO 07-23 03:46:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:05.358893348Z INFO 07-23 03:46:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:10.388523949Z INFO 07-23 03:46:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:15.420504664Z INFO 07-23 03:46:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:20.448707580Z INFO 07-23 03:46:20 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:25.464481774Z INFO 07-23 03:46:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:30.502872201Z INFO 07-23 03:46:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:35.530078168Z INFO 07-23 03:46:35 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:40.559825975Z INFO 07-23 03:46:40 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:45.595368097Z INFO 07-23 03:46:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:50.632083164Z INFO 07-23 03:46:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:55.670123461Z INFO 07-23 03:46:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:00.692064975Z INFO 07-23 03:47:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:05.731449900Z INFO 07-23 03:47:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:10.775862113Z INFO 07-23 03:47:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:15.789807502Z INFO 07-23 03:47:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:18.030344386Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:47:18.267069376Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:47:21.908553681Z INFO 07-23 03:47:21 metrics.py:345] Avg prompt throughput: 1338.8 tokens/s, Avg generation throughput: 7.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:27.607452205Z INFO 07-23 03:47:27 metrics.py:345] Avg prompt throughput: 2156.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:33.592167279Z INFO 07-23 03:47:33 metrics.py:345] Avg prompt throughput: 684.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:38.615627195Z INFO 07-23 03:47:38 metrics.py:345] Avg prompt throughput: 610.3 tokens/s, Avg generation throughput: 8.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:43.652542974Z INFO 07-23 03:47:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:48.069171000Z ERROR 07-23 03:47:48 serving_chat.py:221] Error in applying chat template from request
+2026-07-23T03:47:48.069190665Z ERROR 07-23 03:47:48 serving_chat.py:221] Traceback (most recent call last):
+2026-07-23T03:47:48.069193533Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 188, in create_chat_completion
+2026-07-23T03:47:48.069196058Z ERROR 07-23 03:47:48 serving_chat.py:221]     conversation, mm_data_future = parse_chat_messages_futures(
+2026-07-23T03:47:48.069198382Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 551, in parse_chat_messages_futures
+2026-07-23T03:47:48.069200645Z ERROR 07-23 03:47:48 serving_chat.py:221]     sub_messages = _parse_chat_message_content(msg, mm_tracker)
+2026-07-23T03:47:48.069202837Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 477, in _parse_chat_message_content
+2026-07-23T03:47:48.069204889Z ERROR 07-23 03:47:48 serving_chat.py:221]     result = _parse_chat_message_content_parts(
+2026-07-23T03:47:48.069206940Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 429, in _parse_chat_message_content_parts
+2026-07-23T03:47:48.069209541Z ERROR 07-23 03:47:48 serving_chat.py:221]     mm_parser.parse_image(image_url["url"])
+2026-07-23T03:47:48.069211572Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 311, in parse_image
+2026-07-23T03:47:48.069213819Z ERROR 07-23 03:47:48 serving_chat.py:221]     placeholder = self._tracker.add("image", image_coro)
+2026-07-23T03:47:48.069215830Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 220, in add
+2026-07-23T03:47:48.069218119Z ERROR 07-23 03:47:48 serving_chat.py:221]     raise ValueError(
+2026-07-23T03:47:48.069229290Z ERROR 07-23 03:47:48 serving_chat.py:221] ValueError: At most 1 image(s) may be provided in one request.
+2026-07-23T03:47:48.069293988Z /usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py:222: RuntimeWarning: coroutine 'async_get_and_parse_image' was never awaited
+2026-07-23T03:47:48.069300137Z   return self.create_error_response(str(e))
+2026-07-23T03:47:48.069302341Z RuntimeWarning: Enable tracemalloc to get the object allocation traceback
+2026-07-23T03:47:48.069481495Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:47:48.325722518Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:47:50.228821656Z INFO 07-23 03:47:50 metrics.py:345] Avg prompt throughput: 622.9 tokens/s, Avg generation throughput: 14.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:55.685485411Z INFO 07-23 03:47:55 metrics.py:345] Avg prompt throughput: 2251.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:01.889056918Z INFO 07-23 03:48:01 metrics.py:345] Avg prompt throughput: 1980.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:08.880652390Z INFO 07-23 03:48:08 metrics.py:345] Avg prompt throughput: 1757.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:20.950602402Z INFO 07-23 03:48:20 metrics.py:345] Avg prompt throughput: 547.1 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:25.952959584Z INFO 07-23 03:48:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:30.999396546Z INFO 07-23 03:48:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:36.044256010Z INFO 07-23 03:48:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:41.044599828Z INFO 07-23 03:48:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:46.083073696Z INFO 07-23 03:48:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:51.124521665Z INFO 07-23 03:48:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:56.168277560Z INFO 07-23 03:48:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:01.212516850Z INFO 07-23 03:49:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:06.258871180Z INFO 07-23 03:49:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:11.303607987Z INFO 07-23 03:49:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:16.352861160Z INFO 07-23 03:49:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:21.355919111Z INFO 07-23 03:49:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:23.979457774Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:49:27.605136815Z INFO 07-23 03:49:27 metrics.py:345] Avg prompt throughput: 1310.9 tokens/s, Avg generation throughput: 7.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:33.296245524Z INFO 07-23 03:49:33 metrics.py:345] Avg prompt throughput: 2159.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:38.656394179Z INFO 07-23 03:49:38 metrics.py:345] Avg prompt throughput: 1597.3 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:41.259500869Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:49:44.876448503Z INFO 07-23 03:49:44 metrics.py:345] Avg prompt throughput: 1317.1 tokens/s, Avg generation throughput: 8.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:50.779280791Z INFO 07-23 03:49:50 metrics.py:345] Avg prompt throughput: 1163.2 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:55.782459543Z INFO 07-23 03:49:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:00.821850285Z INFO 07-23 03:50:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:05.860302968Z INFO 07-23 03:50:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:10.896836932Z INFO 07-23 03:50:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:15.907192499Z INFO 07-23 03:50:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:19.131493471Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:50:21.026655055Z INFO 07-23 03:50:21 metrics.py:345] Avg prompt throughput: 800.1 tokens/s, Avg generation throughput: 13.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:26.482545766Z INFO 07-23 03:50:26 metrics.py:345] Avg prompt throughput: 2252.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:32.675695952Z INFO 07-23 03:50:32 metrics.py:345] Avg prompt throughput: 1984.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:37.717894719Z INFO 07-23 03:50:37 metrics.py:345] Avg prompt throughput: 104.5 tokens/s, Avg generation throughput: 13.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:38.747799235Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:50:43.596272691Z INFO 07-23 03:50:43 metrics.py:345] Avg prompt throughput: 1393.6 tokens/s, Avg generation throughput: 3.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:49.494533413Z INFO 07-23 03:50:49 metrics.py:345] Avg prompt throughput: 1317.8 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:54.495926964Z INFO 07-23 03:50:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:59.532585217Z INFO 07-23 03:50:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:51:04.243701758Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:51:04.695557111Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] Exception in worker VllmWorkerProcess while processing method start_worker_execution_loop: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 2 has a total capacty of 31.72 GiB of which 70.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF, Traceback (most recent call last):
+2026-07-23T03:51:04.695590033Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_worker_utils.py", line 224, in _run_worker_process
+2026-07-23T03:51:04.695593398Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = executor(*args, **kwargs)
+2026-07-23T03:51:04.695597304Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.695599822Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
+2026-07-23T03:51:04.695601960Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 69, in start_worker_execution_loop
+2026-07-23T03:51:04.695604422Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = self.execute_model(execute_model_req=None)
+2026-07-23T03:51:04.695606461Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:04.695608857Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = self.model_runner.execute_model(
+2026-07-23T03:51:04.695611175Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.695613359Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
+2026-07-23T03:51:04.695615304Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:04.695617660Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:04.695619640Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695621844Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695623828Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695628609Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695630599Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:04.695632812Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states = self.model(
+2026-07-23T03:51:04.695634801Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695636954Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695639175Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695641326Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695643268Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:04.695645430Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states, residual = layer(
+2026-07-23T03:51:04.695647355Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695649500Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695651500Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695653916Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695655924Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:04.695658142Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:04.695660265Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695662484Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695667765Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695670920Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695672956Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:04.695675139Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:04.695677985Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:04.695680320Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self.forward_native(x, residual)
+2026-07-23T03:51:04.695684136Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:04.695686306Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:04.695688422Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 158, in forward_static
+2026-07-23T03:51:04.695690647Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     variance = x.pow(2).mean(dim=-1, keepdim=True)
+2026-07-23T03:51:04.695693383Z (VllmWorkerProcess pid=345) (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] Exception in worker VllmWorkerProcess while processing method start_worker_execution_loop: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 1 has a total capacty of 31.72 GiB of which 74.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF, Traceback (most recent call last):
+2026-07-23T03:51:04.695697049Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_worker_utils.py", line 224, in _run_worker_process
+2026-07-23T03:51:04.695699616Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = executor(*args, **kwargs)
+2026-07-23T03:51:04.695701640Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.695706279Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
+2026-07-23T03:51:04.695708636Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 69, in start_worker_execution_loop
+2026-07-23T03:51:04.695710796Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = self.execute_model(execute_model_req=None)
+2026-07-23T03:51:04.695712766Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:04.695714957Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = self.model_runner.execute_model(
+2026-07-23T03:51:04.695716932Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.695719271Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
+2026-07-23T03:51:04.695721336Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:04.695723498Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:04.695725472Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695727695Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695729775Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695732069Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695734030Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:04.695736157Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states = self.model(
+2026-07-23T03:51:04.695738121Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695742683Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695744681Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695746918Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695748963Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:04.695751447Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states, residual = layer(
+2026-07-23T03:51:04.695753660Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695755775Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695757874Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695759993Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695762002Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:04.695764118Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:04.695766114Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695768209Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695770214Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695772460Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695774624Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:04.695778783Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:04.695781237Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:04.695783998Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self.forward_native(x, residual)
+2026-07-23T03:51:04.695785906Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:04.695788026Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:04.695790208Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 158, in forward_static
+2026-07-23T03:51:04.695792433Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     variance = x.pow(2).mean(dim=-1, keepdim=True)
+2026-07-23T03:51:04.695794820Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 2 has a total capacty of 31.72 GiB of which 70.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:04.695797659Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] 
+2026-07-23T03:51:04.695799657Z ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 1 has a total capacty of 31.72 GiB of which 74.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:04.695802893Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] 
+2026-07-23T03:51:04.699701286Z ERROR 07-23 03:51:04 async_llm_engine.py:66] Engine background task failed
+2026-07-23T03:51:04.699707282Z ERROR 07-23 03:51:04 async_llm_engine.py:66] Traceback (most recent call last):
+2026-07-23T03:51:04.699709690Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 56, in _log_task_completion
+2026-07-23T03:51:04.699712553Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return_value = task.result()
+2026-07-23T03:51:04.699722044Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 857, in run_engine_loop
+2026-07-23T03:51:04.699724371Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     result = task.result()
+2026-07-23T03:51:04.699726357Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 776, in engine_step
+2026-07-23T03:51:04.699728437Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     request_outputs = await self.engine.step_async(virtual_engine)
+2026-07-23T03:51:04.699730549Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 348, in step_async
+2026-07-23T03:51:04.699732927Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     outputs = await self.model_executor.execute_model_async(
+2026-07-23T03:51:04.699734814Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/distributed_gpu_executor.py", line 181, in execute_model_async
+2026-07-23T03:51:04.699736873Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return await self._driver_execute_model_async(execute_model_req)
+2026-07-23T03:51:04.699738752Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 224, in _driver_execute_model_async
+2026-07-23T03:51:04.699740932Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return await self.driver_exec_model(execute_model_req)
+2026-07-23T03:51:04.699743070Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
+2026-07-23T03:51:04.699745042Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     result = self.fn(*self.args, **self.kwargs)
+2026-07-23T03:51:04.699747316Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:04.699749564Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     output = self.model_runner.execute_model(
+2026-07-23T03:51:04.699751539Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.699753508Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return func(*args, **kwargs)
+2026-07-23T03:51:04.699755607Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:04.699758036Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:04.699760153Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.699762173Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.699764016Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.699768920Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.699770911Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:04.699772926Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     hidden_states = self.model(
+2026-07-23T03:51:04.699774758Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.699776744Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.699778565Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.699780536Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.699782347Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:04.699784609Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     hidden_states, residual = layer(
+2026-07-23T03:51:04.699786447Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.699788829Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.699790628Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.699792704Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.699856811Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:04.699859662Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:04.699861913Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.699864092Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.699866211Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.699868266Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.699870390Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:04.699872478Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:04.699876920Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:04.699879423Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self.forward_native(x, residual)
+2026-07-23T03:51:04.699881751Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:04.699884537Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:04.699886523Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 157, in forward_static
+2026-07-23T03:51:04.699888605Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     x = x.float()
+2026-07-23T03:51:04.699890682Z ERROR 07-23 03:51:04 async_llm_engine.py:66] torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacty of 31.72 GiB of which 50.92 MiB is free. Of the allocated memory 30.86 GiB is allocated by PyTorch, and 210.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:04.700138219Z Exception in callback functools.partial(<function _log_task_completion at 0x7f206fedf1c0>, error_callback=<bound method AsyncLLMEngine._error_callback of <vllm.engine.async_llm_engine.AsyncLLMEngine object at 0x7f206034f010>>)
+2026-07-23T03:51:04.700142915Z handle: <Handle functools.partial(<function _log_task_completion at 0x7f206fedf1c0>, error_callback=<bound method AsyncLLMEngine._error_callback of <vllm.engine.async_llm_engine.AsyncLLMEngine object at 0x7f206034f010>>)>
+2026-07-23T03:51:04.700146529Z Traceback (most recent call last):
+2026-07-23T03:51:04.700148465Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 56, in _log_task_completion
+2026-07-23T03:51:04.700150727Z     return_value = task.result()
+2026-07-23T03:51:04.700152569Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 857, in run_engine_loop
+2026-07-23T03:51:04.700154521Z     result = task.result()
+2026-07-23T03:51:04.700156285Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 776, in engine_step
+2026-07-23T03:51:04.700158380Z     request_outputs = await self.engine.step_async(virtual_engine)
+2026-07-23T03:51:04.700160278Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 348, in step_async
+2026-07-23T03:51:04.700162359Z     outputs = await self.model_executor.execute_model_async(
+2026-07-23T03:51:04.700164171Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/distributed_gpu_executor.py", line 181, in execute_model_async
+2026-07-23T03:51:04.700166399Z     return await self._driver_execute_model_async(execute_model_req)
+2026-07-23T03:51:04.700170872Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 224, in _driver_execute_model_async
+2026-07-23T03:51:04.700172919Z     return await self.driver_exec_model(execute_model_req)
+2026-07-23T03:51:04.700174704Z   File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
+2026-07-23T03:51:04.700176814Z     result = self.fn(*self.args, **self.kwargs)
+2026-07-23T03:51:04.700178633Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:04.700180563Z     output = self.model_runner.execute_model(
+2026-07-23T03:51:04.700182330Z   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.700184286Z     return func(*args, **kwargs)
+2026-07-23T03:51:04.700186064Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:04.700188266Z     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:04.700190129Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.700192031Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.700193904Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.700195807Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.700197556Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:04.700199508Z     hidden_states = self.model(
+2026-07-23T03:51:04.700201263Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.700203336Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.700205463Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.700207334Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.700209080Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:04.700211063Z     hidden_states, residual = layer(
+2026-07-23T03:51:04.700212851Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.700214768Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.700217261Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.700219719Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.700221463Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:04.700223405Z     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:04.700228613Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.700230566Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.700232462Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.700234414Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.700236143Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:04.700238046Z     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:04.700239806Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:04.700241998Z     return self.forward_native(x, residual)
+2026-07-23T03:51:04.700243823Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:04.700245757Z     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:04.700247627Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 157, in forward_static
+2026-07-23T03:51:04.700249543Z     x = x.float()
+2026-07-23T03:51:04.700252832Z torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacty of 31.72 GiB of which 50.92 MiB is free. Of the allocated memory 30.86 GiB is allocated by PyTorch, and 210.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:04.700256610Z 
+2026-07-23T03:51:04.700258408Z The above exception was the direct cause of the following exception:
+2026-07-23T03:51:04.700260214Z 
+2026-07-23T03:51:04.700261917Z Traceback (most recent call last):
+2026-07-23T03:51:04.700263623Z   File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run
+2026-07-23T03:51:04.700265499Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 68, in _log_task_completion
+2026-07-23T03:51:04.700268134Z     raise AsyncEngineDeadError(
+2026-07-23T03:51:04.700269863Z vllm.engine.async_llm_engine.AsyncEngineDeadError: Task finished unexpectedly. This should never happen! Please open an issue on Github. See stack trace above for the actual cause.
+2026-07-23T03:51:04.702552963Z ERROR:    Exception in ASGI application
+2026-07-23T03:51:04.702558866Z Traceback (most recent call last):
+2026-07-23T03:51:04.702560834Z   File "/usr/local/lib/python3.10/site-packages/starlette/_utils.py", line 79, in collapse_excgroups
+2026-07-23T03:51:04.702564379Z     yield
+2026-07-23T03:51:04.702566507Z   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 271, in __call__
+2026-07-23T03:51:04.702571357Z     async with anyio.create_task_group() as task_group:
+2026-07-23T03:51:04.702573227Z   File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 772, in __aexit__
+2026-07-23T03:51:04.702575469Z     raise BaseExceptionGroup(
+2026-07-23T03:51:04.702577231Z exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+2026-07-23T03:51:04.702579140Z 
+2026-07-23T03:51:04.702580984Z During handling of the above exception, another exception occurred:
+2026-07-23T03:51:04.702582850Z 
+2026-07-23T03:51:04.702584544Z Traceback (most recent call last):
+2026-07-23T03:51:04.702586535Z   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
+2026-07-23T03:51:04.702588429Z     await app(scope, receive, sender)
+2026-07-23T03:51:04.702590235Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
+2026-07-23T03:51:04.702592118Z     await response(scope, receive, send)
+2026-07-23T03:51:04.702593877Z   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 270, in __call__
+2026-07-23T03:51:04.702595840Z     with collapse_excgroups():
+2026-07-23T03:51:04.702597561Z   File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
+2026-07-23T03:51:04.702599394Z     self.gen.throw(typ, value, traceback)
+2026-07-23T03:51:04.702601159Z   File "/usr/local/lib/python3.10/site-packages/starlette/_utils.py", line 85, in collapse_excgroups
+2026-07-23T03:51:04.702603068Z     raise exc
+2026-07-23T03:51:04.702604874Z   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 274, in wrap
+2026-07-23T03:51:04.702606737Z     await func()
+2026-07-23T03:51:04.702608502Z   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 254, in stream_response
+2026-07-23T03:51:04.702610428Z     async for chunk in self.body_iterator:
+2026-07-23T03:51:04.702612210Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 846, in chat_completion_stream_generator
+2026-07-23T03:51:04.702614205Z     await self.engine_client.abort(request_id)
+2026-07-23T03:51:04.702615921Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 1243, in abort
+2026-07-23T03:51:04.702617838Z     raise AsyncEngineDeadError(
+2026-07-23T03:51:04.702619831Z vllm.engine.async_llm_engine.AsyncEngineDeadError: Background loop is not running. If it was running, inspect the output to find the stacktrace of the error that caused the background loop to stop (AsyncEngineDeadError).
+2026-07-23T03:51:04.702622281Z 
+2026-07-23T03:51:04.702623917Z The above exception was the direct cause of the following exception:
+2026-07-23T03:51:04.702625718Z 
+2026-07-23T03:51:04.702627946Z Traceback (most recent call last):
+2026-07-23T03:51:04.702643211Z   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
+2026-07-23T03:51:04.702645150Z     await app(scope, receive, sender)
+2026-07-23T03:51:04.702646977Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 716, in __call__
+2026-07-23T03:51:04.702648835Z     await self.middleware_stack(scope, receive, send)
+2026-07-23T03:51:04.702650643Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 736, in app
+2026-07-23T03:51:04.702652476Z     await route.handle(scope, receive, send)
+2026-07-23T03:51:04.702654253Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 290, in handle
+2026-07-23T03:51:04.702656095Z     await self.app(scope, receive, send)
+2026-07-23T03:51:04.702657830Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 78, in app
+2026-07-23T03:51:04.702659880Z     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
+2026-07-23T03:51:04.702662244Z   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 56, in wrapped_app
+2026-07-23T03:51:04.702665694Z     raise RuntimeError("Caught handled exception, but response already started.") from exc
+2026-07-23T03:51:04.702667893Z RuntimeError: Caught handled exception, but response already started.
+2026-07-23T03:51:04.702669731Z 
+2026-07-23T03:51:04.702671340Z The above exception was the direct cause of the following exception:
+2026-07-23T03:51:04.702673239Z 
+2026-07-23T03:51:04.702674862Z Traceback (most recent call last):
+2026-07-23T03:51:04.702677233Z   File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
+2026-07-23T03:51:04.702679161Z     result = await app(  # type: ignore[func-returns-value]
+2026-07-23T03:51:04.702680952Z   File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
+2026-07-23T03:51:04.702682811Z     return await self.app(scope, receive, send)
+2026-07-23T03:51:04.702684557Z   File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 1082, in __call__
+2026-07-23T03:51:04.702686691Z     await super().__call__(scope, receive, send)
+2026-07-23T03:51:04.702688431Z   File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 113, in __call__
+2026-07-23T03:51:04.702690476Z     await self.middleware_stack(scope, receive, send)
+2026-07-23T03:51:04.702692252Z   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
+2026-07-23T03:51:04.702694354Z     raise exc
+2026-07-23T03:51:04.702696382Z   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
+2026-07-23T03:51:04.702698525Z     await self.app(scope, receive, _send)
+2026-07-23T03:51:04.702700285Z   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
+2026-07-23T03:51:04.702705835Z     await self.app(scope, receive, send)
+2026-07-23T03:51:04.702708304Z   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
+2026-07-23T03:51:04.702710205Z     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
+2026-07-23T03:51:04.702712202Z   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 56, in wrapped_app
+2026-07-23T03:51:04.702714127Z     raise RuntimeError("Caught handled exception, but response already started.") from exc
+2026-07-23T03:51:04.702716055Z RuntimeError: Caught handled exception, but response already started.
+2026-07-23T03:51:04.713173377Z CRITICAL 07-23 03:51:04 launcher.py:88] AsyncLLMEngine is already dead, terminating server process
+2026-07-23T03:51:04.714970623Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
+2026-07-23T03:51:04.718637314Z CRITICAL 07-23 03:51:04 launcher.py:88] AsyncLLMEngine is already dead, terminating server process
+2026-07-23T03:51:04.718694821Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
+2026-07-23T03:51:04.722498257Z CRITICAL 07-23 03:51:04 launcher.py:88] AsyncLLMEngine is already dead, terminating server process
+2026-07-23T03:51:04.722585244Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
+2026-07-23T03:51:04.735233268Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:51:04.756590608Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:51:04.768186558Z CRITICAL 07-23 03:51:04 launcher.py:88] AsyncLLMEngine is already dead, terminating server process
+2026-07-23T03:51:04.768268921Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
+2026-07-23T03:51:04.781736020Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:51:04.791948299Z INFO:     Shutting down
+2026-07-23T03:51:04.892200042Z INFO:     Waiting for application shutdown.
+2026-07-23T03:51:04.892346436Z INFO:     Application shutdown complete.
+2026-07-23T03:51:04.893243235Z INFO:     Finished server process [1]
+2026-07-23T03:51:06.253648359Z INFO 07-23 03:51:06 multiproc_worker_utils.py:121] Killing local vLLM worker processes
+2026-07-23T03:51:10.732959761Z Future exception was never retrieved
+2026-07-23T03:51:10.732970009Z future: <Future finished exception=OutOfMemoryError('CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacty of 31.72 GiB of which 50.92 MiB is free. Of the allocated memory 30.86 GiB is allocated by PyTorch, and 210.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF')>
+2026-07-23T03:51:10.732975335Z Traceback (most recent call last):
+2026-07-23T03:51:10.732983365Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 466, in chat_completion_stream_generator
+2026-07-23T03:51:10.732985745Z     async for res in result_generator:
+2026-07-23T03:51:10.732987633Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/utils.py", line 458, in iterate_with_cancellation
+2026-07-23T03:51:10.732989646Z     item = await awaits[0]
+2026-07-23T03:51:10.732991495Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 1046, in generate
+2026-07-23T03:51:10.732993455Z     async for output in await self.add_request(
+2026-07-23T03:51:10.732995398Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 114, in generator
+2026-07-23T03:51:10.732997331Z     raise result
+2026-07-23T03:51:10.732999159Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 56, in _log_task_completion
+2026-07-23T03:51:10.733001089Z     return_value = task.result()
+2026-07-23T03:51:10.733002909Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 857, in run_engine_loop
+2026-07-23T03:51:10.733004845Z     result = task.result()
+2026-07-23T03:51:10.733006732Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 776, in engine_step
+2026-07-23T03:51:10.733008659Z     request_outputs = await self.engine.step_async(virtual_engine)
+2026-07-23T03:51:10.733010626Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 348, in step_async
+2026-07-23T03:51:10.733012560Z     outputs = await self.model_executor.execute_model_async(
+2026-07-23T03:51:10.733014593Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/distributed_gpu_executor.py", line 181, in execute_model_async
+2026-07-23T03:51:10.733016583Z     return await self._driver_execute_model_async(execute_model_req)
+2026-07-23T03:51:10.733018783Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 224, in _driver_execute_model_async
+2026-07-23T03:51:10.733020797Z     return await self.driver_exec_model(execute_model_req)
+2026-07-23T03:51:10.733022570Z   File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
+2026-07-23T03:51:10.733024502Z     result = self.fn(*self.args, **self.kwargs)
+2026-07-23T03:51:10.733026262Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:10.733028655Z     output = self.model_runner.execute_model(
+2026-07-23T03:51:10.733030678Z   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:10.733032715Z     return func(*args, **kwargs)
+2026-07-23T03:51:10.733034659Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:10.733039337Z     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:10.733041153Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:10.733043068Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:10.733044905Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:10.733046777Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:10.733050243Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:10.733053412Z     hidden_states = self.model(
+2026-07-23T03:51:10.733057551Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:10.733059640Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:10.733061352Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:10.733063252Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:10.733065023Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:10.733067165Z     hidden_states, residual = layer(
+2026-07-23T03:51:10.733069092Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:10.733071000Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:10.733072984Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:10.733074867Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:10.733076607Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:10.733078748Z     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:10.733080781Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:10.733082662Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:10.733084425Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:10.733086295Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:10.733088148Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:10.733090040Z     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:10.733091807Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:10.733093814Z     return self.forward_native(x, residual)
+2026-07-23T03:51:10.733095539Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:10.733101307Z     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:10.733104565Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 157, in forward_static
+2026-07-23T03:51:10.733107808Z     x = x.float()
+2026-07-23T03:51:10.733109873Z torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacty of 31.72 GiB of which 50.92 MiB is free. Of the allocated memory 30.86 GiB is allocated by PyTorch, and 210.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:10.735532351Z Task exception was never retrieved
+2026-07-23T03:51:10.735545848Z future: <Task finished name='Task-129921' coro=<MultiprocessingGPUExecutorAsync._start_worker_execution_loop() done, defined at /usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py:253> exception=OutOfMemoryError('CUDA out of memory. Tried to allocate 32.00 MiB. GPU 2 has a total capacty of 31.72 GiB of which 70.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF')>
+2026-07-23T03:51:10.735550322Z Traceback (most recent call last):
+2026-07-23T03:51:10.735552351Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 258, in _start_worker_execution_loop
+2026-07-23T03:51:10.735554626Z     return await asyncio.gather(*coros)
+2026-07-23T03:51:10.735556397Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_worker_utils.py", line 183, in execute_method_async
+2026-07-23T03:51:10.735558462Z     return await future
+2026-07-23T03:51:10.735560338Z torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 2 has a total capacty of 31.72 GiB of which 70.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T02:29:15.072467456Z (VllmWorkerProcess pid=346) INFO 07-23 02:29:15 model_runner.py:1074] Starting to load model /model...
+2026-07-23T02:29:15.123205314Z (VllmWorkerProcess pid=344) INFO 07-23 02:29:15 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:15.123647472Z (VllmWorkerProcess pid=346) INFO 07-23 02:29:15 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:15.136250183Z INFO 07-23 02:29:15 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:15.142620520Z (VllmWorkerProcess pid=345) INFO 07-23 02:29:15 selector.py:115] Using XFormers backend.
+2026-07-23T02:29:15.267061728Z 
+Loading safetensors checkpoint shards:   0% Completed | 0/16 [00:00<?, ?it/s]
+2026-07-23T02:29:16.504035279Z 
+Loading safetensors checkpoint shards:   6% Completed | 1/16 [00:01<00:18,  1.24s/it]
+2026-07-23T02:29:17.560515507Z 
+Loading safetensors checkpoint shards:  12% Completed | 2/16 [00:02<00:15,  1.13s/it]
+2026-07-23T02:29:18.454967452Z 
+Loading safetensors checkpoint shards:  19% Completed | 3/16 [00:03<00:13,  1.02s/it]
+2026-07-23T02:29:19.622930900Z 
+Loading safetensors checkpoint shards:  25% Completed | 4/16 [00:04<00:12,  1.08s/it]
+2026-07-23T02:29:20.607005016Z 
+Loading safetensors checkpoint shards:  31% Completed | 5/16 [00:05<00:11,  1.05s/it]
+2026-07-23T02:29:21.663620433Z 
+Loading safetensors checkpoint shards:  38% Completed | 6/16 [00:06<00:10,  1.05s/it]
+2026-07-23T02:29:22.762559585Z 
+Loading safetensors checkpoint shards:  44% Completed | 7/16 [00:07<00:09,  1.07s/it]
+2026-07-23T02:29:23.920513395Z 
+Loading safetensors checkpoint shards:  50% Completed | 8/16 [00:08<00:08,  1.09s/it]
+2026-07-23T02:29:25.327235482Z 
+Loading safetensors checkpoint shards:  56% Completed | 9/16 [00:10<00:08,  1.19s/it]
+2026-07-23T02:29:26.537079790Z 
+Loading safetensors checkpoint shards:  62% Completed | 10/16 [00:11<00:07,  1.20s/it]
+2026-07-23T02:29:27.538590658Z 
+Loading safetensors checkpoint shards:  69% Completed | 11/16 [00:12<00:05,  1.14s/it]
+2026-07-23T02:29:28.909915510Z 
+Loading safetensors checkpoint shards:  75% Completed | 12/16 [00:13<00:04,  1.21s/it]
+2026-07-23T02:29:30.177921768Z 
+Loading safetensors checkpoint shards:  81% Completed | 13/16 [00:14<00:03,  1.23s/it]
+2026-07-23T02:29:31.144529015Z 
+Loading safetensors checkpoint shards:  88% Completed | 14/16 [00:15<00:02,  1.15s/it]
+2026-07-23T02:29:32.170218873Z 
+Loading safetensors checkpoint shards:  94% Completed | 15/16 [00:16<00:01,  1.11s/it]
+2026-07-23T02:29:33.339901076Z 
+Loading safetensors checkpoint shards: 100% Completed | 16/16 [00:18<00:00,  1.13s/it]
+2026-07-23T02:29:33.340059995Z 
+Loading safetensors checkpoint shards: 100% Completed | 16/16 [00:18<00:00,  1.13s/it]
+2026-07-23T02:29:33.340067531Z 
+2026-07-23T02:29:33.791755457Z (VllmWorkerProcess pid=345) INFO 07-23 02:29:33 model_runner.py:1085] Loading model weights took 17.3529 GB
+2026-07-23T02:29:33.791764238Z (VllmWorkerProcess pid=344) INFO 07-23 02:29:33 model_runner.py:1085] Loading model weights took 17.3529 GB
+2026-07-23T02:29:33.792212417Z (VllmWorkerProcess pid=346) INFO 07-23 02:29:33 model_runner.py:1085] Loading model weights took 17.3529 GB
+2026-07-23T02:29:33.799049642Z INFO 07-23 02:29:33 model_runner.py:1085] Loading model weights took 17.3529 GB
+2026-07-23T02:32:25.902136731Z (VllmWorkerProcess pid=346) INFO 07-23 02:32:25 corex_gdn.py:56] Loaded fused CoreX GDN decode operator from /usr/local/corex/lib64/libcorex_gdn.so
+2026-07-23T02:32:25.903895709Z (VllmWorkerProcess pid=345) INFO 07-23 02:32:25 corex_gdn.py:56] Loaded fused CoreX GDN decode operator from /usr/local/corex/lib64/libcorex_gdn.so
+2026-07-23T02:32:25.905723523Z (VllmWorkerProcess pid=346) INFO 07-23 02:32:25 corex_gdn.py:228] Using fused CoreX GDN prefill operator
+2026-07-23T02:32:25.907479743Z (VllmWorkerProcess pid=345) INFO 07-23 02:32:25 corex_gdn.py:228] Using fused CoreX GDN prefill operator
+2026-07-23T02:32:25.915921136Z INFO 07-23 02:32:25 corex_gdn.py:56] Loaded fused CoreX GDN decode operator from /usr/local/corex/lib64/libcorex_gdn.so
+2026-07-23T02:32:25.918179567Z (VllmWorkerProcess pid=344) INFO 07-23 02:32:25 corex_gdn.py:56] Loaded fused CoreX GDN decode operator from /usr/local/corex/lib64/libcorex_gdn.so
+2026-07-23T02:32:25.919624182Z INFO 07-23 02:32:25 corex_gdn.py:228] Using fused CoreX GDN prefill operator
+2026-07-23T02:32:25.921789252Z (VllmWorkerProcess pid=344) INFO 07-23 02:32:25 corex_gdn.py:228] Using fused CoreX GDN prefill operator
+2026-07-23T02:32:25.950838770Z (VllmWorkerProcess pid=346) INFO 07-23 02:32:25 corex_moe.py:339] Using CoreX fused MoE prefill operator: tokens=4096, kernel=expert-grouped-wmma
+2026-07-23T02:32:25.950882505Z (VllmWorkerProcess pid=344) INFO 07-23 02:32:25 corex_moe.py:339] Using CoreX fused MoE prefill operator: tokens=4096, kernel=expert-grouped-wmma
+2026-07-23T02:32:25.950908225Z INFO 07-23 02:32:25 corex_moe.py:339] Using CoreX fused MoE prefill operator: tokens=4096, kernel=expert-grouped-wmma
+2026-07-23T02:32:25.950931658Z (VllmWorkerProcess pid=345) INFO 07-23 02:32:25 corex_moe.py:339] Using CoreX fused MoE prefill operator: tokens=4096, kernel=expert-grouped-wmma
+2026-07-23T02:32:26.045895062Z (VllmWorkerProcess pid=346) INFO 07-23 02:32:26 corex_fa2.py:333] Using CoreX FA2 packed prefill: B=2 Hq=4 Hkv=1 D=256 max_q=2048 max_k=2048
+2026-07-23T02:32:26.045990261Z (VllmWorkerProcess pid=345) INFO 07-23 02:32:26 corex_fa2.py:333] Using CoreX FA2 packed prefill: B=2 Hq=4 Hkv=1 D=256 max_q=2048 max_k=2048
+2026-07-23T02:32:26.046000888Z (VllmWorkerProcess pid=344) INFO 07-23 02:32:26 corex_fa2.py:333] Using CoreX FA2 packed prefill: B=2 Hq=4 Hkv=1 D=256 max_q=2048 max_k=2048
+2026-07-23T02:32:26.046049182Z INFO 07-23 02:32:26 corex_fa2.py:333] Using CoreX FA2 packed prefill: B=2 Hq=4 Hkv=1 D=256 max_q=2048 max_k=2048
+2026-07-23T02:32:27.731056211Z INFO 07-23 02:32:27 distributed_gpu_executor.py:57] # GPU blocks: 19259, # CPU blocks: 6553
+2026-07-23T02:32:27.731106894Z INFO 07-23 02:32:27 distributed_gpu_executor.py:61] Maximum concurrency for 256000 tokens per request: 1.20x
+2026-07-23T02:32:31.498383100Z INFO 07-23 02:32:31 serving_chat.py:107] "auto" tool choice has been enabled please note that while the parallel_tool_calls client option is preset for compatibility reasons, it will be ignored.
+2026-07-23T02:32:31.499707957Z INFO 07-23 02:32:31 serving_chat.py:129] Reasoning parser 'qwen3' enabled.
+2026-07-23T02:32:31.499774658Z WARNING 07-23 02:32:31 serving_embedding.py:199] embedding_mode is False. Embedding API will not work.
+2026-07-23T02:32:31.499825780Z INFO 07-23 02:32:31 launcher.py:19] Available routes are:
+2026-07-23T02:32:31.499867858Z INFO 07-23 02:32:31 launcher.py:27] Route: /openapi.json, Methods: HEAD, GET
+2026-07-23T02:32:31.499886381Z INFO 07-23 02:32:31 launcher.py:27] Route: /docs, Methods: HEAD, GET
+2026-07-23T02:32:31.499904662Z INFO 07-23 02:32:31 launcher.py:27] Route: /docs/oauth2-redirect, Methods: HEAD, GET
+2026-07-23T02:32:31.499945605Z INFO 07-23 02:32:31 launcher.py:27] Route: /redoc, Methods: HEAD, GET
+2026-07-23T02:32:31.499951625Z INFO 07-23 02:32:31 launcher.py:27] Route: /health, Methods: GET
+2026-07-23T02:32:31.499989247Z INFO 07-23 02:32:31 launcher.py:27] Route: /tokenize, Methods: POST
+2026-07-23T02:32:31.499991563Z INFO 07-23 02:32:31 launcher.py:27] Route: /detokenize, Methods: POST
+2026-07-23T02:32:31.500017919Z INFO 07-23 02:32:31 launcher.py:27] Route: /v1/models, Methods: GET
+2026-07-23T02:32:31.500027643Z INFO 07-23 02:32:31 launcher.py:27] Route: /version, Methods: GET
+2026-07-23T02:32:31.500047221Z INFO 07-23 02:32:31 launcher.py:27] Route: /v1/chat/completions, Methods: POST
+2026-07-23T02:32:31.500071390Z INFO 07-23 02:32:31 launcher.py:27] Route: /v1/completions, Methods: POST
+2026-07-23T02:32:31.500084135Z INFO 07-23 02:32:31 launcher.py:27] Route: /v1/embeddings, Methods: POST
+2026-07-23T02:32:31.517724005Z INFO:     Started server process [1]
+2026-07-23T02:32:31.517775355Z INFO:     Waiting for application startup.
+2026-07-23T02:32:31.518104234Z INFO:     Application startup complete.
+2026-07-23T02:32:31.518434320Z INFO:     Uvicorn running on socket ('0.0.0.0', 8000) (Press CTRL+C to quit)
+2026-07-23T02:32:41.518080032Z INFO 07-23 02:32:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:32:44.976380698Z INFO:     172.17.0.1:34848 - "GET /health HTTP/1.1" 200 OK
+2026-07-23T02:32:44.977131890Z INFO:     172.17.0.1:34848 - "GET /v1/models HTTP/1.1" 200 OK
+2026-07-23T02:32:51.517794437Z INFO 07-23 02:32:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:01.518176592Z INFO 07-23 02:33:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:11.517753255Z INFO 07-23 02:33:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:21.518188707Z INFO 07-23 02:33:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:31.518515223Z INFO 07-23 02:33:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:41.518939596Z INFO 07-23 02:33:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:33:51.519425845Z INFO 07-23 02:33:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:01.519896974Z INFO 07-23 02:34:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:11.520418085Z INFO 07-23 02:34:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:21.520940369Z INFO 07-23 02:34:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:31.521337663Z INFO 07-23 02:34:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:41.521711416Z INFO 07-23 02:34:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:34:51.522131821Z INFO 07-23 02:34:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:01.522501314Z INFO 07-23 02:35:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:11.522857298Z INFO 07-23 02:35:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:21.523285280Z INFO 07-23 02:35:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:31.522702429Z INFO 07-23 02:35:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:41.523044729Z INFO 07-23 02:35:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:35:51.523462651Z INFO 07-23 02:35:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:01.523885993Z INFO 07-23 02:36:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:11.524260127Z INFO 07-23 02:36:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:21.523723862Z INFO 07-23 02:36:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:31.524139239Z INFO 07-23 02:36:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:41.523592375Z INFO 07-23 02:36:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:36:51.523995257Z INFO 07-23 02:36:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:01.524428577Z INFO 07-23 02:37:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:11.525008725Z INFO 07-23 02:37:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:21.524752003Z INFO 07-23 02:37:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:31.525171202Z INFO 07-23 02:37:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:35.774341598Z INFO:     172.28.22.23:22372 - "GET /get_model_info HTTP/1.1" 404 Not Found
+2026-07-23T02:37:35.776361569Z INFO:     172.28.22.23:43360 - "GET /get_server_info HTTP/1.1" 404 Not Found
+2026-07-23T02:37:35.777756478Z INFO:     172.28.22.23:30571 - "GET /version HTTP/1.1" 200 OK
+2026-07-23T02:37:36.257909619Z /usr/local/lib/python3.10/site-packages/pyairports/airports.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
+2026-07-23T02:37:36.257934560Z   from pkg_resources import resource_string
+2026-07-23T02:37:37.889916076Z (VllmWorkerProcess pid=346) INFO 07-23 02:37:37 corex_fa2.py:507] Using CoreX paged FA2 chunked prefill: B=1 Hq=4 Hkv=1 D=256 max_q=17 cache_blocks=2
+2026-07-23T02:37:37.889927964Z INFO 07-23 02:37:37 corex_fa2.py:507] Using CoreX paged FA2 chunked prefill: B=1 Hq=4 Hkv=1 D=256 max_q=17 cache_blocks=2
+2026-07-23T02:37:37.889989104Z (VllmWorkerProcess pid=344) INFO 07-23 02:37:37 corex_fa2.py:507] Using CoreX paged FA2 chunked prefill: B=1 Hq=4 Hkv=1 D=256 max_q=17 cache_blocks=2
+2026-07-23T02:37:37.889993203Z (VllmWorkerProcess pid=345) INFO 07-23 02:37:37 corex_fa2.py:507] Using CoreX paged FA2 chunked prefill: B=1 Hq=4 Hkv=1 D=256 max_q=17 cache_blocks=2
+2026-07-23T02:37:37.970977734Z INFO 07-23 02:37:37 metrics.py:345] Avg prompt throughput: 2.6 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:37.977175350Z (VllmWorkerProcess pid=346) INFO 07-23 02:37:37 corex_gdn.py:138] Using fused CoreX GDN decode operator
+2026-07-23T02:37:37.977199535Z (VllmWorkerProcess pid=344) INFO 07-23 02:37:37 corex_gdn.py:138] Using fused CoreX GDN decode operator
+2026-07-23T02:37:37.977216621Z INFO 07-23 02:37:37 corex_gdn.py:138] Using fused CoreX GDN decode operator
+2026-07-23T02:37:37.977220191Z (VllmWorkerProcess pid=345) INFO 07-23 02:37:37 corex_gdn.py:138] Using fused CoreX GDN decode operator
+2026-07-23T02:37:37.977862426Z (VllmWorkerProcess pid=346) INFO 07-23 02:37:37 corex_moe.py:249] Using CoreX fused MoE decode operator
+2026-07-23T02:37:37.977866130Z (VllmWorkerProcess pid=345) INFO 07-23 02:37:37 corex_moe.py:249] Using CoreX fused MoE decode operator
+2026-07-23T02:37:37.977868711Z INFO 07-23 02:37:37 corex_moe.py:249] Using CoreX fused MoE decode operator
+2026-07-23T02:37:37.977871546Z (VllmWorkerProcess pid=344) INFO 07-23 02:37:37 corex_moe.py:249] Using CoreX fused MoE decode operator
+2026-07-23T02:37:43.000160753Z INFO 07-23 02:37:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:44.269161765Z INFO:     172.28.22.23:13116 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:37:44.396724776Z INFO:     172.28.22.23:7057 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:37:48.010705708Z INFO 07-23 02:37:48 metrics.py:345] Avg prompt throughput: 57.9 tokens/s, Avg generation throughput: 16.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:49.136976286Z INFO:     172.28.22.23:45259 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:37:53.025290553Z INFO 07-23 02:37:53 metrics.py:345] Avg prompt throughput: 7.2 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:37:58.061514748Z INFO 07-23 02:37:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:03.073788209Z INFO 07-23 02:38:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:06.918087930Z INFO:     172.28.22.23:34227 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:38:08.377827090Z INFO 07-23 02:38:08 metrics.py:345] Avg prompt throughput: 4.3 tokens/s, Avg generation throughput: 15.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:13.046591181Z INFO:     172.28.22.23:29692 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:38:13.172739732Z INFO:     172.28.22.23:48540 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:38:13.710792259Z INFO 07-23 02:38:13 metrics.py:345] Avg prompt throughput: 58.3 tokens/s, Avg generation throughput: 18.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:14.666143261Z INFO:     172.28.22.23:34379 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:38:18.720975352Z INFO 07-23 02:38:18 metrics.py:345] Avg prompt throughput: 74.2 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:23.725539749Z INFO 07-23 02:38:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:28.760348109Z INFO 07-23 02:38:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:33.763160416Z INFO 07-23 02:38:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:38.786603579Z INFO 07-23 02:38:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:43.822945213Z INFO 07-23 02:38:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:48.829240975Z INFO 07-23 02:38:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:53.861865865Z INFO 07-23 02:38:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:38:58.900819345Z INFO 07-23 02:38:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:03.930669117Z INFO 07-23 02:39:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:08.932255487Z INFO 07-23 02:39:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:13.947463156Z INFO 07-23 02:39:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:17.129519291Z INFO:     172.28.22.23:47285 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:21.526611923Z INFO 07-23 02:39:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 8.7 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:22.789790337Z ERROR 07-23 02:39:22 serving_chat.py:227] Error in loading multi-modal data
+2026-07-23T02:39:22.789821427Z ERROR 07-23 02:39:22 serving_chat.py:227] Traceback (most recent call last):
+2026-07-23T02:39:22.789824505Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 770, in _request
+2026-07-23T02:39:22.789827021Z ERROR 07-23 02:39:22 serving_chat.py:227]     resp = await handler(req)
+2026-07-23T02:39:22.789829231Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 725, in _connect_and_send_request
+2026-07-23T02:39:22.789831554Z ERROR 07-23 02:39:22 serving_chat.py:227]     conn = await self._connector.connect(
+2026-07-23T02:39:22.789833480Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 642, in connect
+2026-07-23T02:39:22.789835993Z ERROR 07-23 02:39:22 serving_chat.py:227]     proto = await self._create_connection(req, traces, timeout)
+2026-07-23T02:39:22.789838507Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1209, in _create_connection
+2026-07-23T02:39:22.789840743Z ERROR 07-23 02:39:22 serving_chat.py:227]     _, proto = await self._create_direct_connection(req, traces, timeout)
+2026-07-23T02:39:22.789842704Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1550, in _create_direct_connection
+2026-07-23T02:39:22.789844833Z ERROR 07-23 02:39:22 serving_chat.py:227]     transp, proto = await self._wrap_create_connection(
+2026-07-23T02:39:22.789857869Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1283, in _wrap_create_connection
+2026-07-23T02:39:22.789859986Z ERROR 07-23 02:39:22 serving_chat.py:227]     return await self._loop.create_connection(*args, **kwargs, sock=sock)
+2026-07-23T02:39:22.789861916Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "uvloop/loop.pyx", line 2088, in create_connection
+2026-07-23T02:39:22.789864184Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "uvloop/loop.pyx", line 2083, in uvloop.loop.Loop.create_connection
+2026-07-23T02:39:22.789866163Z ERROR 07-23 02:39:22 serving_chat.py:227] asyncio.exceptions.CancelledError
+2026-07-23T02:39:22.789867986Z ERROR 07-23 02:39:22 serving_chat.py:227] 
+2026-07-23T02:39:22.789878123Z ERROR 07-23 02:39:22 serving_chat.py:227] The above exception was the direct cause of the following exception:
+2026-07-23T02:39:22.789887569Z ERROR 07-23 02:39:22 serving_chat.py:227] 
+2026-07-23T02:39:22.789890593Z ERROR 07-23 02:39:22 serving_chat.py:227] Traceback (most recent call last):
+2026-07-23T02:39:22.789892686Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 225, in create_chat_completion
+2026-07-23T02:39:22.789894923Z ERROR 07-23 02:39:22 serving_chat.py:227]     mm_data = await mm_data_future
+2026-07-23T02:39:22.789896983Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 248, in all_mm_data
+2026-07-23T02:39:22.789899260Z ERROR 07-23 02:39:22 serving_chat.py:227]     items = await asyncio.gather(*self._items)
+2026-07-23T02:39:22.789901151Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/multimodal/utils.py", line 140, in async_get_and_parse_image
+2026-07-23T02:39:22.789903256Z ERROR 07-23 02:39:22 serving_chat.py:227]     image = await async_fetch_image(image_url)
+2026-07-23T02:39:22.789905277Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/multimodal/utils.py", line 62, in async_fetch_image
+2026-07-23T02:39:22.789907335Z ERROR 07-23 02:39:22 serving_chat.py:227]     image_raw = await global_http_connection.async_get_bytes(
+2026-07-23T02:39:22.789909226Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/corex/lib/python3/dist-packages/vllm/connections.py", line 92, in async_get_bytes
+2026-07-23T02:39:22.789911211Z ERROR 07-23 02:39:22 serving_chat.py:227]     async with await self.get_async_response(url, timeout=timeout) as r:
+2026-07-23T02:39:22.789913906Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 1488, in __aenter__
+2026-07-23T02:39:22.789915849Z ERROR 07-23 02:39:22 serving_chat.py:227]     self._resp: _RetType = await self._coro
+2026-07-23T02:39:22.789918323Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 622, in _request
+2026-07-23T02:39:22.789921398Z ERROR 07-23 02:39:22 serving_chat.py:227]     with timer:
+2026-07-23T02:39:22.789929127Z ERROR 07-23 02:39:22 serving_chat.py:227]   File "/usr/local/lib/python3.10/site-packages/aiohttp/helpers.py", line 685, in __exit__
+2026-07-23T02:39:22.789931189Z ERROR 07-23 02:39:22 serving_chat.py:227]     raise asyncio.TimeoutError from exc_val
+2026-07-23T02:39:22.789933347Z ERROR 07-23 02:39:22 serving_chat.py:227] asyncio.exceptions.TimeoutError
+2026-07-23T02:39:22.789982206Z INFO:     172.28.22.23:63905 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:39:23.339790176Z INFO:     172.28.22.23:48915 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:23.862218623Z INFO:     172.28.22.23:39188 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:26.537243563Z INFO 07-23 02:39:26 metrics.py:345] Avg prompt throughput: 12.0 tokens/s, Avg generation throughput: 12.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:31.709534191Z INFO:     172.28.22.23:30519 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:31.709975900Z INFO 07-23 02:39:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:32.540087790Z INFO:     172.28.22.23:28182 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:36.711389406Z INFO 07-23 02:39:36 metrics.py:345] Avg prompt throughput: 6.0 tokens/s, Avg generation throughput: 20.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:38.834910565Z INFO:     172.28.22.23:25497 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:40.106985905Z INFO:     172.28.22.23:31187 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:41.375383434Z INFO:     172.28.22.23:1054 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:41.736972408Z INFO 07-23 02:39:41 metrics.py:345] Avg prompt throughput: 6.6 tokens/s, Avg generation throughput: 17.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:42.644170272Z INFO:     172.28.22.23:10283 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:43.909888974Z INFO:     172.28.22.23:43195 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:44.026597768Z INFO:     172.28.22.23:31826 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:39:45.279682681Z INFO:     172.28.22.23:31890 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:46.532238457Z INFO:     172.28.22.23:52470 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:46.768052149Z INFO 07-23 02:39:46 metrics.py:345] Avg prompt throughput: 8.7 tokens/s, Avg generation throughput: 15.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:47.779232275Z INFO:     172.28.22.23:1935 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:47.894850660Z INFO:     172.28.22.23:24799 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:39:49.140115276Z INFO:     172.28.22.23:32610 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:50.385833050Z INFO:     172.28.22.23:34946 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:51.632711881Z INFO:     172.28.22.23:29640 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:51.824060918Z INFO 07-23 02:39:51 metrics.py:345] Avg prompt throughput: 8.7 tokens/s, Avg generation throughput: 15.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:52.883677883Z INFO:     172.28.22.23:25348 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:54.134965370Z INFO:     172.28.22.23:37574 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:55.381846551Z INFO:     172.28.22.23:9523 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:56.629565470Z INFO:     172.28.22.23:56394 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:39:56.827523945Z INFO 07-23 02:39:56 metrics.py:345] Avg prompt throughput: 8.8 tokens/s, Avg generation throughput: 16.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:39:58.129748260Z INFO:     172.28.22.23:9093 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:40:01.848026951Z INFO 07-23 02:40:01 metrics.py:345] Avg prompt throughput: 2.8 tokens/s, Avg generation throughput: 23.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:06.858910882Z INFO 07-23 02:40:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:11.862489179Z INFO 07-23 02:40:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:16.899163580Z INFO 07-23 02:40:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:21.910058417Z INFO 07-23 02:40:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:26.929209213Z INFO 07-23 02:40:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:31.962337057Z INFO 07-23 02:40:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:36.964040774Z INFO 07-23 02:40:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:41.993035873Z INFO 07-23 02:40:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:43.624005510Z INFO:     172.28.22.23:2587 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:40:44.061237847Z INFO:     172.28.22.23:6084 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:40:47.185504184Z INFO:     172.28.22.23:50569 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:40:47.377278073Z INFO 07-23 02:40:47 metrics.py:345] Avg prompt throughput: 7.8 tokens/s, Avg generation throughput: 17.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:52.412861124Z INFO 07-23 02:40:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:40:57.415707222Z INFO 07-23 02:40:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:02.437584264Z INFO 07-23 02:41:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:07.476637249Z INFO 07-23 02:41:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:12.517176365Z INFO 07-23 02:41:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:17.531413276Z INFO 07-23 02:41:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:22.564441104Z INFO 07-23 02:41:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:27.577151702Z INFO 07-23 02:41:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:32.611584807Z INFO 07-23 02:41:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:37.623284038Z INFO 07-23 02:41:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:42.639031894Z INFO 07-23 02:41:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:47.641862259Z INFO 07-23 02:41:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:52.683325904Z INFO 07-23 02:41:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:41:57.720443718Z INFO 07-23 02:41:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:02.748390108Z INFO 07-23 02:42:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:07.784213915Z INFO 07-23 02:42:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:12.827245969Z INFO 07-23 02:42:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:17.859544046Z INFO 07-23 02:42:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:22.889429150Z INFO 07-23 02:42:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:27.923212049Z INFO 07-23 02:42:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:32.925156292Z INFO 07-23 02:42:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:37.955994920Z INFO 07-23 02:42:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:40.293610439Z INFO:     172.28.22.23:17026 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:40.411301190Z INFO:     172.28.22.23:47159 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:42:40.524865477Z INFO:     172.28.22.23:53783 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:42:40.637743018Z INFO:     172.28.22.23:24159 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:42:41.181611635Z INFO:     172.28.22.23:38467 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:41.727889239Z INFO:     172.28.22.23:24963 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:42.962296089Z INFO 07-23 02:42:42 metrics.py:345] Avg prompt throughput: 64.9 tokens/s, Avg generation throughput: 14.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:42:43.783197309Z INFO:     172.28.22.23:14284 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:44.459669799Z INFO:     172.28.22.23:49964 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:44.575803355Z INFO:     172.28.22.23:54360 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:42:47.973780549Z INFO 07-23 02:42:47 metrics.py:345] Avg prompt throughput: 15.0 tokens/s, Avg generation throughput: 18.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:01.858799064Z INFO 07-23 02:43:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:10.553432764Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:04<00:09,  4.87s/it]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:04<00:00,  1.31s/it]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:05<00:00,  1.69s/it]
+2026-07-23T02:43:10.569991401Z INFO 07-23 02:43:10 metrics.py:345] Avg prompt throughput: 4.0 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:11.990491527Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.23it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.17it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  6.98it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.52it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.10it/s]
+2026-07-23T02:43:13.445065774Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.22it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.22it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  7.07it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.63it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.14it/s]
+2026-07-23T02:43:14.876448068Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.23it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.28it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  7.15it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.73it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.23it/s]
+2026-07-23T02:43:15.701717222Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.19it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.19it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.69it/s]
+2026-07-23T02:43:15.720137025Z INFO 07-23 02:43:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:16.473274583Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.21it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  5.71it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.28it/s]
+2026-07-23T02:43:18.904638207Z 
+Compiling FSM index for all state transitions:   0%|          | 0/23 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   4%|▍         | 1/23 [00:00<00:10,  2.14it/s]
+Compiling FSM index for all state transitions:  13%|█▎        | 3/23 [00:00<00:04,  4.13it/s]
+Compiling FSM index for all state transitions:  22%|██▏       | 5/23 [00:00<00:02,  6.94it/s]
+Compiling FSM index for all state transitions:  30%|███       | 7/23 [00:00<00:01,  9.55it/s]
+Compiling FSM index for all state transitions:  39%|███▉      | 9/23 [00:01<00:01, 11.88it/s]
+Compiling FSM index for all state transitions:  48%|████▊     | 11/23 [00:01<00:00, 13.84it/s]
+Compiling FSM index for all state transitions:  57%|█████▋    | 13/23 [00:01<00:00, 15.41it/s]
+Compiling FSM index for all state transitions:  65%|██████▌   | 15/23 [00:01<00:00, 16.62it/s]
+Compiling FSM index for all state transitions:  74%|███████▍  | 17/23 [00:01<00:00, 17.54it/s]
+Compiling FSM index for all state transitions:  87%|████████▋ | 20/23 [00:01<00:00, 18.47it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:01<00:00, 19.06it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:02<00:00,  9.89it/s]
+2026-07-23T02:43:21.350197196Z 
+Compiling FSM index for all state transitions:   0%|          | 0/23 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   4%|▍         | 1/23 [00:00<00:10,  2.14it/s]
+Compiling FSM index for all state transitions:  13%|█▎        | 3/23 [00:00<00:04,  4.24it/s]
+Compiling FSM index for all state transitions:  22%|██▏       | 5/23 [00:00<00:02,  7.05it/s]
+Compiling FSM index for all state transitions:  30%|███       | 7/23 [00:00<00:01,  9.67it/s]
+Compiling FSM index for all state transitions:  39%|███▉      | 9/23 [00:01<00:01, 11.99it/s]
+Compiling FSM index for all state transitions:  52%|█████▏    | 12/23 [00:01<00:00, 14.60it/s]
+Compiling FSM index for all state transitions:  61%|██████    | 14/23 [00:01<00:00, 15.87it/s]
+Compiling FSM index for all state transitions:  74%|███████▍  | 17/23 [00:01<00:00, 17.26it/s]
+Compiling FSM index for all state transitions:  87%|████████▋ | 20/23 [00:01<00:00, 18.16it/s]
+Compiling FSM index for all state transitions:  96%|█████████▌| 22/23 [00:01<00:00, 18.40it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:02<00:00,  9.89it/s]
+2026-07-23T02:43:21.397139672Z INFO 07-23 02:43:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:22.261063391Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.22it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.24it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.73it/s]
+2026-07-23T02:43:22.957955678Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.25it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.31it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.77it/s]
+2026-07-23T02:43:24.351851677Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.25it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.32it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  7.21it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.80it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.26it/s]
+2026-07-23T02:43:25.800354211Z 
+Compiling FSM index for all state transitions:   0%|          | 0/7 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 1/7 [00:00<00:02,  2.16it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 3/7 [00:00<00:00,  4.28it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 5/7 [00:00<00:00,  7.15it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:00<00:00,  9.73it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 7/7 [00:01<00:00,  5.16it/s]
+2026-07-23T02:43:26.709113498Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.23it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.25it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.73it/s]
+2026-07-23T02:43:26.727308641Z INFO 07-23 02:43:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:27.404963736Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.25it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.30it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.78it/s]
+2026-07-23T02:43:29.815689817Z 
+Compiling FSM index for all state transitions:   0%|          | 0/23 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   4%|▍         | 1/23 [00:00<00:10,  2.14it/s]
+Compiling FSM index for all state transitions:  13%|█▎        | 3/23 [00:00<00:04,  4.21it/s]
+Compiling FSM index for all state transitions:  22%|██▏       | 5/23 [00:00<00:02,  7.04it/s]
+Compiling FSM index for all state transitions:  30%|███       | 7/23 [00:00<00:01,  9.64it/s]
+Compiling FSM index for all state transitions:  39%|███▉      | 9/23 [00:01<00:01, 11.95it/s]
+Compiling FSM index for all state transitions:  48%|████▊     | 11/23 [00:01<00:00, 13.88it/s]
+Compiling FSM index for all state transitions:  57%|█████▋    | 13/23 [00:01<00:00, 15.42it/s]
+Compiling FSM index for all state transitions:  65%|██████▌   | 15/23 [00:01<00:00, 16.60it/s]
+Compiling FSM index for all state transitions:  74%|███████▍  | 17/23 [00:01<00:00, 17.53it/s]
+Compiling FSM index for all state transitions:  87%|████████▋ | 20/23 [00:01<00:00, 18.46it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:01<00:00, 18.81it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:02<00:00,  9.91it/s]
+2026-07-23T02:43:31.881017891Z INFO 07-23 02:43:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:32.258419554Z 
+Compiling FSM index for all state transitions:   0%|          | 0/23 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   4%|▍         | 1/23 [00:00<00:10,  2.13it/s]
+Compiling FSM index for all state transitions:  13%|█▎        | 3/23 [00:00<00:04,  4.25it/s]
+Compiling FSM index for all state transitions:  22%|██▏       | 5/23 [00:00<00:02,  7.10it/s]
+Compiling FSM index for all state transitions:  30%|███       | 7/23 [00:00<00:01,  9.72it/s]
+Compiling FSM index for all state transitions:  39%|███▉      | 9/23 [00:01<00:01, 12.03it/s]
+Compiling FSM index for all state transitions:  48%|████▊     | 11/23 [00:01<00:00, 13.97it/s]
+Compiling FSM index for all state transitions:  57%|█████▋    | 13/23 [00:01<00:00, 15.50it/s]
+Compiling FSM index for all state transitions:  65%|██████▌   | 15/23 [00:01<00:00, 16.69it/s]
+Compiling FSM index for all state transitions:  74%|███████▍  | 17/23 [00:01<00:00, 17.58it/s]
+Compiling FSM index for all state transitions:  83%|████████▎ | 19/23 [00:01<00:00, 18.25it/s]
+Compiling FSM index for all state transitions:  96%|█████████▌| 22/23 [00:01<00:00, 18.93it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 23/23 [00:02<00:00,  9.89it/s]
+2026-07-23T02:43:32.983539758Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.24it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.29it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.76it/s]
+2026-07-23T02:43:33.672113037Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.25it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.32it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.78it/s]
+2026-07-23T02:43:34.368060101Z 
+Compiling FSM index for all state transitions:   0%|          | 0/3 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  33%|███▎      | 1/3 [00:00<00:00,  2.24it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  6.28it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 3/3 [00:00<00:00,  4.76it/s]
+2026-07-23T02:43:34.941584523Z 
+Compiling FSM index for all state transitions:   0%|          | 0/2 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:  50%|█████     | 1/2 [00:00<00:00,  2.25it/s]
+Compiling FSM index for all state transitions:  50%|█████     | 1/2 [00:00<00:00,  1.96it/s]
+2026-07-23T02:43:35.230436592Z INFO:     172.28.22.23:18839 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:38.310835204Z 
+Compiling FSM index for all state transitions:   0%|          | 0/35 [00:00<?, ?it/s]
+Compiling FSM index for all state transitions:   3%|▎         | 1/35 [00:00<00:16,  2.11it/s]
+Compiling FSM index for all state transitions:   9%|▊         | 3/35 [00:00<00:05,  6.13it/s]
+Compiling FSM index for all state transitions:  14%|█▍        | 5/35 [00:00<00:03,  9.43it/s]
+Compiling FSM index for all state transitions:  20%|██        | 7/35 [00:00<00:02, 12.06it/s]
+Compiling FSM index for all state transitions:  26%|██▌       | 9/35 [00:00<00:01, 13.89it/s]
+Compiling FSM index for all state transitions:  31%|███▏      | 11/35 [00:01<00:01, 15.18it/s]
+Compiling FSM index for all state transitions:  37%|███▋      | 13/35 [00:01<00:02, 10.52it/s]
+Compiling FSM index for all state transitions:  43%|████▎     | 15/35 [00:01<00:01, 12.25it/s]
+Compiling FSM index for all state transitions:  49%|████▊     | 17/35 [00:01<00:01, 13.80it/s]
+Compiling FSM index for all state transitions:  54%|█████▍    | 19/35 [00:01<00:01, 14.75it/s]
+Compiling FSM index for all state transitions:  60%|██████    | 21/35 [00:01<00:00, 15.66it/s]
+Compiling FSM index for all state transitions:  66%|██████▌   | 23/35 [00:01<00:00, 16.67it/s]
+Compiling FSM index for all state transitions:  71%|███████▏  | 25/35 [00:01<00:00, 17.44it/s]
+Compiling FSM index for all state transitions:  77%|███████▋  | 27/35 [00:02<00:00, 17.73it/s]
+Compiling FSM index for all state transitions:  83%|████████▎ | 29/35 [00:02<00:00, 17.93it/s]
+Compiling FSM index for all state transitions:  89%|████████▊ | 31/35 [00:02<00:00, 18.39it/s]
+Compiling FSM index for all state transitions:  94%|█████████▍| 33/35 [00:02<00:00, 18.07it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 35/35 [00:02<00:00, 18.50it/s]
+Compiling FSM index for all state transitions: 100%|██████████| 35/35 [00:02<00:00, 11.94it/s]
+2026-07-23T02:43:38.462680310Z INFO 07-23 02:43:38 metrics.py:345] Avg prompt throughput: 3.5 tokens/s, Avg generation throughput: 0.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:39.428082640Z INFO:     172.28.22.23:41311 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:41.354629107Z INFO:     172.28.22.23:35141 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:41.951932899Z INFO:     172.28.22.23:40937 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:42.708616569Z INFO:     172.28.22.23:3677 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:43.378056127Z INFO:     172.28.22.23:46863 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:43.593932801Z INFO 07-23 02:43:43 metrics.py:345] Avg prompt throughput: 22.6 tokens/s, Avg generation throughput: 13.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:43:44.482046916Z INFO:     172.28.22.23:46268 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:43:51.952117180Z INFO 07-23 02:43:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 1.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:01.952434751Z INFO 07-23 02:44:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:11.953116612Z INFO 07-23 02:44:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:21.953465417Z INFO 07-23 02:44:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:31.954404947Z INFO 07-23 02:44:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:41.955602923Z INFO 07-23 02:44:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:44:51.956600553Z INFO 07-23 02:44:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:01.429424131Z INFO 07-23 02:45:01 metrics.py:345] Avg prompt throughput: 33.4 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:02.020271472Z INFO:     172.28.22.23:10993 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:45:02.023173066Z INFO:     172.28.22.23:34691 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:45:02.542033396Z INFO:     172.28.22.23:44245 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:45:03.031598393Z INFO:     172.28.22.23:5495 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:45:03.034079308Z INFO:     172.28.22.23:33361 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:45:03.035816648Z INFO:     172.28.22.23:1279 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:45:03.038855639Z ERROR 07-23 02:45:03 serving_chat.py:221] Error in applying chat template from request
+2026-07-23T02:45:03.038865861Z ERROR 07-23 02:45:03 serving_chat.py:221] Traceback (most recent call last):
+2026-07-23T02:45:03.038868246Z ERROR 07-23 02:45:03 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 210, in create_chat_completion
+2026-07-23T02:45:03.038870615Z ERROR 07-23 02:45:03 serving_chat.py:221]     prompt = apply_hf_chat_template(
+2026-07-23T02:45:03.038872879Z ERROR 07-23 02:45:03 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 574, in apply_hf_chat_template
+2026-07-23T02:45:03.038875232Z ERROR 07-23 02:45:03 serving_chat.py:221]     return tokenizer.apply_chat_template(
+2026-07-23T02:45:03.038877096Z ERROR 07-23 02:45:03 serving_chat.py:221]   File "/usr/local/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1623, in apply_chat_template
+2026-07-23T02:45:03.038879145Z ERROR 07-23 02:45:03 serving_chat.py:221]     isinstance(conversation[0], (list, tuple)) or hasattr(conversation[0], "messages")
+2026-07-23T02:45:03.038881206Z ERROR 07-23 02:45:03 serving_chat.py:221] IndexError: list index out of range
+2026-07-23T02:45:03.039113159Z /usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/api_server.py:316: RuntimeWarning: coroutine 'AsyncMultiModalItemTracker.all_mm_data' was never awaited
+2026-07-23T02:45:03.039116978Z   generator = await chat(raw_request).create_chat_completion(
+2026-07-23T02:45:03.039119691Z RuntimeWarning: Enable tracemalloc to get the object allocation traceback
+2026-07-23T02:45:03.039249660Z INFO:     172.28.22.23:39148 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:45:12.020995535Z INFO 07-23 02:45:12 metrics.py:345] Avg prompt throughput: 3.6 tokens/s, Avg generation throughput: 1.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:18.489898901Z INFO:     172.28.22.23:47283 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:45:18.707020154Z INFO 07-23 02:45:18 metrics.py:345] Avg prompt throughput: 5.8 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:23.747292153Z INFO 07-23 02:45:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:28.783886861Z INFO 07-23 02:45:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:33.805881768Z INFO 07-23 02:45:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:38.828813167Z INFO 07-23 02:45:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:43.837844599Z INFO 07-23 02:45:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:48.863019622Z INFO 07-23 02:45:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:53.902955303Z INFO 07-23 02:45:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:45:58.919192631Z INFO 07-23 02:45:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:03.934701110Z INFO 07-23 02:46:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:08.965144483Z INFO 07-23 02:46:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:13.969498143Z INFO 07-23 02:46:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:19.000136074Z INFO 07-23 02:46:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:24.035080024Z INFO 07-23 02:46:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:29.054704172Z INFO 07-23 02:46:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:34.075751226Z INFO 07-23 02:46:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:39.109012197Z INFO 07-23 02:46:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:44.142536722Z INFO 07-23 02:46:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:49.151030740Z INFO 07-23 02:46:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:54.155952934Z INFO 07-23 02:46:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:46:59.168320199Z INFO 07-23 02:46:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:04.198850701Z INFO 07-23 02:47:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:09.227769526Z INFO 07-23 02:47:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:14.236849600Z INFO 07-23 02:47:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:19.250060510Z INFO 07-23 02:47:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:24.278224796Z INFO 07-23 02:47:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:29.302744374Z INFO 07-23 02:47:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:34.340169258Z INFO 07-23 02:47:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:39.373263942Z INFO 07-23 02:47:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:44.400614703Z INFO 07-23 02:47:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:49.429303585Z INFO 07-23 02:47:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:54.454631971Z INFO 07-23 02:47:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:47:59.476270315Z INFO 07-23 02:47:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:04.489781001Z INFO 07-23 02:48:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:09.502845316Z INFO 07-23 02:48:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:14.525253044Z INFO 07-23 02:48:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:19.535460002Z INFO 07-23 02:48:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:24.549336142Z INFO 07-23 02:48:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:29.562276558Z INFO 07-23 02:48:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:34.582693092Z INFO 07-23 02:48:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:39.606402177Z INFO 07-23 02:48:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:44.618006935Z INFO 07-23 02:48:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:49.639887326Z INFO 07-23 02:48:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:54.661553164Z INFO 07-23 02:48:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:48:59.691872658Z INFO 07-23 02:48:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:04.716442787Z INFO 07-23 02:49:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:09.737600900Z INFO 07-23 02:49:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:14.765495628Z INFO 07-23 02:49:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:19.785966586Z INFO 07-23 02:49:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:24.808905236Z INFO 07-23 02:49:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:29.823237965Z INFO 07-23 02:49:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:34.840213253Z INFO 07-23 02:49:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:39.860991050Z INFO 07-23 02:49:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:44.886401826Z INFO 07-23 02:49:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:49.902498154Z INFO 07-23 02:49:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:54.932337177Z INFO 07-23 02:49:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:49:59.972912007Z INFO 07-23 02:49:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:05.007985393Z INFO 07-23 02:50:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:10.048282976Z INFO 07-23 02:50:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:15.051923596Z INFO 07-23 02:50:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.2%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:20.093869860Z INFO 07-23 02:50:20 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:25.125545341Z INFO 07-23 02:50:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:30.161930016Z INFO 07-23 02:50:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:35.204345197Z INFO 07-23 02:50:35 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:40.208651498Z INFO 07-23 02:50:40 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:45.211397957Z INFO 07-23 02:50:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:50.215514007Z INFO 07-23 02:50:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:50:55.226202583Z INFO 07-23 02:50:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:00.234263458Z INFO 07-23 02:51:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:05.268080221Z INFO 07-23 02:51:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:10.295027960Z INFO 07-23 02:51:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:15.333710639Z INFO 07-23 02:51:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:22.023882201Z INFO 07-23 02:51:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 6.1 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:32.024428280Z INFO 07-23 02:51:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:42.025051809Z INFO 07-23 02:51:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:51:52.025669180Z INFO 07-23 02:51:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:02.026245267Z INFO 07-23 02:52:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:12.025670280Z INFO 07-23 02:52:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:22.025991502Z INFO 07-23 02:52:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:32.026096441Z INFO 07-23 02:52:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:42.026445024Z INFO 07-23 02:52:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:52:52.026881786Z INFO 07-23 02:52:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:02.027443741Z INFO 07-23 02:53:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:12.027857295Z INFO 07-23 02:53:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:22.021717002Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:22.028015831Z INFO 07-23 02:53:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:22.034636325Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:22.042292538Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:22.049601298Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:22.237627231Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:53:27.059387000Z INFO 07-23 02:53:27 metrics.py:345] Avg prompt throughput: 1478.3 tokens/s, Avg generation throughput: 1.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:32.069675497Z INFO 07-23 02:53:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:37.075811510Z INFO 07-23 02:53:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:41.384843095Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:53:41.531870163Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:53:42.183280911Z INFO 07-23 02:53:42 metrics.py:345] Avg prompt throughput: 179.7 tokens/s, Avg generation throughput: 19.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:47.184870989Z INFO 07-23 02:53:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:52.202464890Z INFO 07-23 02:53:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:53:57.216053216Z INFO 07-23 02:53:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:02.236765351Z INFO 07-23 02:54:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:07.245326134Z INFO 07-23 02:54:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:12.257378000Z INFO 07-23 02:54:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:17.265723592Z INFO 07-23 02:54:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:22.274426444Z INFO 07-23 02:54:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:27.288180475Z INFO 07-23 02:54:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:32.303355245Z INFO 07-23 02:54:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:37.308955789Z INFO 07-23 02:54:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:42.316230767Z INFO 07-23 02:54:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:47.330731343Z INFO 07-23 02:54:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:52.350938716Z INFO 07-23 02:54:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:54:57.368955403Z INFO 07-23 02:54:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:02.387972119Z INFO 07-23 02:55:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:07.403238913Z INFO 07-23 02:55:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:12.424487516Z INFO 07-23 02:55:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:17.468209611Z INFO 07-23 02:55:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:22.468838506Z INFO 07-23 02:55:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:27.495305182Z INFO 07-23 02:55:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:31.663705730Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:55:33.559722894Z INFO 07-23 02:55:33 metrics.py:345] Avg prompt throughput: 675.4 tokens/s, Avg generation throughput: 14.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:39.017972246Z INFO 07-23 02:55:39 metrics.py:345] Avg prompt throughput: 2251.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:45.229906291Z INFO 07-23 02:55:45 metrics.py:345] Avg prompt throughput: 1978.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:52.031344888Z INFO 07-23 02:55:52 metrics.py:345] Avg prompt throughput: 1204.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:55:56.488143481Z (VllmWorkerProcess pid=346) INFO 07-23 02:55:56 corex_fa2.py:225] Using CoreX paged decode: B=1 Hq=4 Hkv=1 D=256 max_k=45455 partition=256
+2026-07-23T02:55:56.488439965Z INFO 07-23 02:55:56 corex_fa2.py:225] Using CoreX paged decode: B=1 Hq=4 Hkv=1 D=256 max_k=45455 partition=256
+2026-07-23T02:55:56.488465183Z (VllmWorkerProcess pid=345) INFO 07-23 02:55:56 corex_fa2.py:225] Using CoreX paged decode: B=1 Hq=4 Hkv=1 D=256 max_k=45455 partition=256
+2026-07-23T02:55:56.488468721Z (VllmWorkerProcess pid=344) INFO 07-23 02:55:56 corex_fa2.py:225] Using CoreX paged decode: B=1 Hq=4 Hkv=1 D=256 max_k=45455 partition=256
+2026-07-23T02:55:57.074355971Z INFO 07-23 02:55:57 metrics.py:345] Avg prompt throughput: 1703.4 tokens/s, Avg generation throughput: 2.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:02.075744305Z INFO 07-23 02:56:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:04.960717007Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:05.133613133Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:56:07.092301075Z INFO 07-23 02:56:07 metrics.py:345] Avg prompt throughput: 664.6 tokens/s, Avg generation throughput: 12.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:12.126688361Z INFO 07-23 02:56:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:14.381995529Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:14.389331161Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:14.395208292Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:14.402701762Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:14.562022048Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:56:18.164082686Z INFO 07-23 02:56:18 metrics.py:345] Avg prompt throughput: 1356.9 tokens/s, Avg generation throughput: 8.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:23.187795865Z INFO 07-23 02:56:23 metrics.py:345] Avg prompt throughput: 620.6 tokens/s, Avg generation throughput: 7.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:28.216477560Z INFO 07-23 02:56:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:32.311129833Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:56:32.500362238Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:56:34.377488148Z INFO 07-23 02:56:34 metrics.py:345] Avg prompt throughput: 664.8 tokens/s, Avg generation throughput: 14.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:39.826490851Z INFO 07-23 02:56:39 metrics.py:345] Avg prompt throughput: 2255.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:44.855411293Z INFO 07-23 02:56:44 metrics.py:345] Avg prompt throughput: 1032.6 tokens/s, Avg generation throughput: 2.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:49.884515182Z INFO 07-23 02:56:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:56:54.030501607Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:56:55.903654883Z INFO 07-23 02:56:55 metrics.py:345] Avg prompt throughput: 680.5 tokens/s, Avg generation throughput: 14.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:01.492620867Z INFO 07-23 02:57:01 metrics.py:345] Avg prompt throughput: 1348.2 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:06.518263412Z INFO 07-23 02:57:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:06.756437999Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:57:12.034175081Z INFO 07-23 02:57:12 metrics.py:345] Avg prompt throughput: 1485.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:18.176599373Z INFO 07-23 02:57:18 metrics.py:345] Avg prompt throughput: 2667.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:24.913022642Z INFO 07-23 02:57:24 metrics.py:345] Avg prompt throughput: 1824.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:32.034111195Z INFO 07-23 02:57:32 metrics.py:345] Avg prompt throughput: 1150.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:42.034654469Z INFO 07-23 02:57:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:47.041274003Z INFO 07-23 02:57:47 metrics.py:345] Avg prompt throughput: 804.5 tokens/s, Avg generation throughput: 7.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.9%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:51.606342181Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:57:51.804362711Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:57:53.676250964Z INFO 07-23 02:57:53 metrics.py:345] Avg prompt throughput: 617.3 tokens/s, Avg generation throughput: 13.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:57:59.365803779Z INFO 07-23 02:57:59 metrics.py:345] Avg prompt throughput: 1703.3 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:04.393685506Z INFO 07-23 02:58:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:04.609577440Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:04.615677229Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:04.798268401Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:58:10.219376350Z INFO 07-23 02:58:10 metrics.py:345] Avg prompt throughput: 2109.3 tokens/s, Avg generation throughput: 0.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:15.231494329Z INFO 07-23 02:58:15 metrics.py:345] Avg prompt throughput: 266.8 tokens/s, Avg generation throughput: 13.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:16.102093564Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:16.116114529Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:16.122546146Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:58:16.358547897Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:58:21.798309841Z INFO 07-23 02:58:21 metrics.py:345] Avg prompt throughput: 1871.3 tokens/s, Avg generation throughput: 2.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:27.758005289Z INFO 07-23 02:58:27 metrics.py:345] Avg prompt throughput: 2061.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:34.479363188Z INFO 07-23 02:58:34 metrics.py:345] Avg prompt throughput: 1828.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:41.994543666Z INFO 07-23 02:58:41 metrics.py:345] Avg prompt throughput: 1635.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.0%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:46.793774434Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:58:48.658622582Z INFO 07-23 02:58:48 metrics.py:345] Avg prompt throughput: 631.3 tokens/s, Avg generation throughput: 11.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:53.685872484Z INFO 07-23 02:58:53 metrics.py:345] Avg prompt throughput: 764.6 tokens/s, Avg generation throughput: 8.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:58:58.727252608Z INFO 07-23 02:58:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:01.342048508Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T02:59:01.502531224Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:59:05.905542940Z INFO 07-23 02:59:05 metrics.py:345] Avg prompt throughput: 1007.1 tokens/s, Avg generation throughput: 8.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:10.916217422Z INFO 07-23 02:59:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:15.938840078Z INFO 07-23 02:59:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:16.919636112Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:59:22.003161588Z INFO 07-23 02:59:22 metrics.py:345] Avg prompt throughput: 1326.5 tokens/s, Avg generation throughput: 3.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:27.012505251Z INFO 07-23 02:59:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:31.000804350Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:59:32.036163466Z INFO 07-23 02:59:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 17.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:37.075964266Z INFO 07-23 02:59:37 metrics.py:345] Avg prompt throughput: 1428.0 tokens/s, Avg generation throughput: 7.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:42.093095363Z INFO 07-23 02:59:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:44.048812647Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T02:59:47.672900564Z INFO 07-23 02:59:47 metrics.py:345] Avg prompt throughput: 1468.2 tokens/s, Avg generation throughput: 6.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:53.382936907Z INFO 07-23 02:59:53 metrics.py:345] Avg prompt throughput: 2152.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.1%, CPU KV cache usage: 0.0%.
+2026-07-23T02:59:59.847651377Z INFO 07-23 02:59:59 metrics.py:345] Avg prompt throughput: 1900.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:12.037275698Z INFO 07-23 03:00:12 metrics.py:345] Avg prompt throughput: 336.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:17.039468718Z INFO 07-23 03:00:17 metrics.py:345] Avg prompt throughput: 728.1 tokens/s, Avg generation throughput: 12.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 13.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:20.892612495Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:00:22.757922531Z INFO 07-23 03:00:22 metrics.py:345] Avg prompt throughput: 716.3 tokens/s, Avg generation throughput: 12.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:27.762148011Z INFO 07-23 03:00:27 metrics.py:345] Avg prompt throughput: 990.8 tokens/s, Avg generation throughput: 9.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:32.765102431Z INFO 07-23 03:00:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:37.783726602Z INFO 07-23 03:00:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:41.716682007Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:00:43.588916994Z INFO 07-23 03:00:43 metrics.py:345] Avg prompt throughput: 705.6 tokens/s, Avg generation throughput: 14.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:48.604975727Z INFO 07-23 03:00:48 metrics.py:345] Avg prompt throughput: 877.8 tokens/s, Avg generation throughput: 12.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:53.636064513Z INFO 07-23 03:00:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:00:58.643811919Z INFO 07-23 03:00:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:03.437865787Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:01:04.011675373Z INFO 07-23 03:01:04 metrics.py:345] Avg prompt throughput: 134.5 tokens/s, Avg generation throughput: 19.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:09.032151825Z INFO 07-23 03:01:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:14.076641438Z INFO 07-23 03:01:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:19.116791927Z INFO 07-23 03:01:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:24.132816105Z INFO 07-23 03:01:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:29.152510624Z INFO 07-23 03:01:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:34.164408415Z INFO 07-23 03:01:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:39.177405529Z INFO 07-23 03:01:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:44.196112311Z INFO 07-23 03:01:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:49.218177392Z INFO 07-23 03:01:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:54.260843117Z INFO 07-23 03:01:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:01:59.263581709Z INFO 07-23 03:01:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:04.286227306Z INFO 07-23 03:02:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:09.307925835Z INFO 07-23 03:02:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:14.334136745Z INFO 07-23 03:02:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:19.371566476Z INFO 07-23 03:02:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:24.403118388Z INFO 07-23 03:02:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:29.442979636Z INFO 07-23 03:02:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:34.460345650Z INFO 07-23 03:02:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:34.943043684Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:02:39.543923430Z INFO 07-23 03:02:39 metrics.py:345] Avg prompt throughput: 1440.3 tokens/s, Avg generation throughput: 1.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:43.567655344Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:02:43.733772678Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:02:45.627591234Z INFO 07-23 03:02:45 metrics.py:345] Avg prompt throughput: 673.3 tokens/s, Avg generation throughput: 14.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:50.087651530Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:02:50.253227675Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:02:52.041399788Z INFO 07-23 03:02:52 metrics.py:345] Avg prompt throughput: 556.3 tokens/s, Avg generation throughput: 10.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:02:57.051443405Z INFO 07-23 03:02:57 metrics.py:345] Avg prompt throughput: 1924.7 tokens/s, Avg generation throughput: 7.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:02.093327354Z INFO 07-23 03:03:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:07.102889862Z INFO 07-23 03:03:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:07.595301247Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:03:12.607251038Z INFO 07-23 03:03:12 metrics.py:345] Avg prompt throughput: 1430.3 tokens/s, Avg generation throughput: 1.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:14.974811119Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:03:20.025072743Z INFO 07-23 03:03:20 metrics.py:345] Avg prompt throughput: 1075.1 tokens/s, Avg generation throughput: 6.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:22.619326919Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:03:26.117782285Z INFO 07-23 03:03:26 metrics.py:345] Avg prompt throughput: 1344.6 tokens/s, Avg generation throughput: 8.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 8.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:31.802449723Z INFO 07-23 03:03:31 metrics.py:345] Avg prompt throughput: 2161.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 8.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:40.400462617Z INFO 07-23 03:03:40 metrics.py:345] Avg prompt throughput: 786.3 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 8.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:40.860411102Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:03:46.589024290Z INFO 07-23 03:03:46 metrics.py:345] Avg prompt throughput: 1610.1 tokens/s, Avg generation throughput: 1.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:51.589997032Z INFO 07-23 03:03:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:56.633889073Z INFO 07-23 03:03:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:03:57.579527268Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:03:57.746631502Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:04:02.045624284Z INFO 07-23 03:04:02 metrics.py:345] Avg prompt throughput: 756.9 tokens/s, Avg generation throughput: 3.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:04.802148446Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:04:05.027520323Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:04:08.674733439Z INFO 07-23 03:04:08 metrics.py:345] Avg prompt throughput: 1849.3 tokens/s, Avg generation throughput: 6.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:14.390439518Z INFO 07-23 03:04:14 metrics.py:345] Avg prompt throughput: 2149.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:20.860969569Z INFO 07-23 03:04:20 metrics.py:345] Avg prompt throughput: 1899.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:28.091096635Z INFO 07-23 03:04:28 metrics.py:345] Avg prompt throughput: 1699.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:33.666207138Z INFO 07-23 03:04:33 metrics.py:345] Avg prompt throughput: 832.6 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:37.651979949Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:04:38.689261053Z INFO 07-23 03:04:38 metrics.py:345] Avg prompt throughput: 108.1 tokens/s, Avg generation throughput: 17.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:43.719316648Z INFO 07-23 03:04:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:48.726194909Z INFO 07-23 03:04:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:53.773772705Z INFO 07-23 03:04:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:04:58.783851267Z INFO 07-23 03:04:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:03.796039763Z INFO 07-23 03:05:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:08.800370314Z INFO 07-23 03:05:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:13.817925374Z INFO 07-23 03:05:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:18.827534721Z INFO 07-23 03:05:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:23.836345906Z INFO 07-23 03:05:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:28.867665562Z INFO 07-23 03:05:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:33.879535202Z INFO 07-23 03:05:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:38.887930697Z INFO 07-23 03:05:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:43.911415061Z INFO 07-23 03:05:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:48.919604312Z INFO 07-23 03:05:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:53.923735935Z INFO 07-23 03:05:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:05:58.937847984Z INFO 07-23 03:05:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:03.976626873Z INFO 07-23 03:06:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:08.988814054Z INFO 07-23 03:06:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:13.996690644Z INFO 07-23 03:06:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:19.022685216Z INFO 07-23 03:06:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:24.035035366Z INFO 07-23 03:06:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:29.045998728Z INFO 07-23 03:06:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:34.065591058Z INFO 07-23 03:06:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:39.065937087Z INFO 07-23 03:06:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:44.089850754Z INFO 07-23 03:06:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:49.122224783Z INFO 07-23 03:06:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:54.140258854Z INFO 07-23 03:06:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:06:59.155775804Z INFO 07-23 03:06:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:04.167641219Z INFO 07-23 03:07:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:09.183289842Z INFO 07-23 03:07:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:14.190459782Z INFO 07-23 03:07:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:16.063329774Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:07:19.693310580Z INFO 07-23 03:07:19 metrics.py:345] Avg prompt throughput: 1488.7 tokens/s, Avg generation throughput: 6.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:24.709589906Z INFO 07-23 03:07:24 metrics.py:345] Avg prompt throughput: 289.5 tokens/s, Avg generation throughput: 15.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:29.752714538Z INFO 07-23 03:07:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:34.754784357Z INFO 07-23 03:07:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:37.046291229Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:07:40.663717101Z INFO 07-23 03:07:40 metrics.py:345] Avg prompt throughput: 1386.4 tokens/s, Avg generation throughput: 8.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:45.698396136Z INFO 07-23 03:07:45 metrics.py:345] Avg prompt throughput: 1079.9 tokens/s, Avg generation throughput: 5.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:50.739722502Z INFO 07-23 03:07:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:55.759442021Z INFO 07-23 03:07:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:07:59.292075461Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:08:00.760756817Z INFO 07-23 03:08:00 metrics.py:345] Avg prompt throughput: 68.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:05.803486926Z INFO 07-23 03:08:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:10.807800210Z INFO 07-23 03:08:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:15.810120875Z INFO 07-23 03:08:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:20.821109568Z INFO 07-23 03:08:20 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:25.834563489Z INFO 07-23 03:08:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:30.852701551Z INFO 07-23 03:08:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:35.895279345Z INFO 07-23 03:08:35 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:40.898125953Z INFO 07-23 03:08:40 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:45.903238274Z INFO 07-23 03:08:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:50.911176380Z INFO 07-23 03:08:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:08:55.948442038Z INFO 07-23 03:08:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:00.986020640Z INFO 07-23 03:09:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:05.987833861Z INFO 07-23 03:09:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:10.999189231Z INFO 07-23 03:09:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:16.010005913Z INFO 07-23 03:09:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:21.023415174Z INFO 07-23 03:09:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:26.037961024Z INFO 07-23 03:09:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:31.059518782Z INFO 07-23 03:09:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:36.076762327Z INFO 07-23 03:09:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:41.098050215Z INFO 07-23 03:09:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:46.109931334Z INFO 07-23 03:09:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:51.117802613Z INFO 07-23 03:09:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:09:52.198541671Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:09:58.473656203Z INFO 07-23 03:09:58 metrics.py:345] Avg prompt throughput: 1424.5 tokens/s, Avg generation throughput: 2.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:03.483554227Z INFO 07-23 03:10:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:08.486025664Z INFO 07-23 03:10:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:13.387287348Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:10:13.573126997Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:10:15.451842926Z INFO 07-23 03:10:15 metrics.py:345] Avg prompt throughput: 588.0 tokens/s, Avg generation throughput: 15.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:20.484742816Z INFO 07-23 03:10:20 metrics.py:345] Avg prompt throughput: 1315.2 tokens/s, Avg generation throughput: 2.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:25.490834189Z INFO 07-23 03:10:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:30.496012765Z INFO 07-23 03:10:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:33.890420869Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:10:35.776362721Z INFO 07-23 03:10:35 metrics.py:345] Avg prompt throughput: 775.7 tokens/s, Avg generation throughput: 13.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:40.799476081Z INFO 07-23 03:10:40 metrics.py:345] Avg prompt throughput: 834.3 tokens/s, Avg generation throughput: 13.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:42.302078280Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:10:45.904718727Z INFO 07-23 03:10:45 metrics.py:345] Avg prompt throughput: 1604.6 tokens/s, Avg generation throughput: 5.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:50.918196398Z INFO 07-23 03:10:50 metrics.py:345] Avg prompt throughput: 43.5 tokens/s, Avg generation throughput: 20.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:10:51.861459428Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:10:51.868181568Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:10:52.095460779Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:10:57.543735685Z INFO 07-23 03:10:57 metrics.py:345] Avg prompt throughput: 1854.6 tokens/s, Avg generation throughput: 3.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:03.506535562Z INFO 07-23 03:11:03 metrics.py:345] Avg prompt throughput: 2060.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:10.241413771Z INFO 07-23 03:11:10 metrics.py:345] Avg prompt throughput: 1824.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:17.765981052Z INFO 07-23 03:11:17 metrics.py:345] Avg prompt throughput: 1633.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:22.772423832Z INFO 07-23 03:11:22 metrics.py:345] Avg prompt throughput: 173.2 tokens/s, Avg generation throughput: 3.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:27.820717941Z INFO 07-23 03:11:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:32.868761969Z INFO 07-23 03:11:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 16.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:35.822104096Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:11:39.432629061Z INFO 07-23 03:11:39 metrics.py:345] Avg prompt throughput: 1248.1 tokens/s, Avg generation throughput: 8.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:42.539380381Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:11:46.145813361Z INFO 07-23 03:11:46 metrics.py:345] Avg prompt throughput: 1271.2 tokens/s, Avg generation throughput: 7.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:50.849891660Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:11:50.859074820Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:11:50.870425879Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:11:51.098867253Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:11:52.776745737Z INFO 07-23 03:11:52 metrics.py:345] Avg prompt throughput: 71.2 tokens/s, Avg generation throughput: 12.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:57.791435469Z INFO 07-23 03:11:57 metrics.py:345] Avg prompt throughput: 1751.5 tokens/s, Avg generation throughput: 10.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:11:58.518847773Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:11:58.688843761Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:12:04.113810544Z INFO 07-23 03:12:04 metrics.py:345] Avg prompt throughput: 1943.6 tokens/s, Avg generation throughput: 2.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:09.124640595Z INFO 07-23 03:12:09 metrics.py:345] Avg prompt throughput: 450.4 tokens/s, Avg generation throughput: 7.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:14.096840115Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:12:14.320275730Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:12:16.080406340Z INFO 07-23 03:12:16 metrics.py:345] Avg prompt throughput: 588.9 tokens/s, Avg generation throughput: 16.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:21.516014892Z INFO 07-23 03:12:21 metrics.py:345] Avg prompt throughput: 2260.7 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:32.515215161Z INFO 07-23 03:12:32 metrics.py:345] Avg prompt throughput: 1016.9 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:33.019477144Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:12:41.016003493Z INFO 07-23 03:12:41 metrics.py:345] Avg prompt throughput: 1415.5 tokens/s, Avg generation throughput: 0.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:46.059272778Z INFO 07-23 03:12:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:51.061081047Z INFO 07-23 03:12:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:56.072726396Z INFO 07-23 03:12:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:12:58.413592732Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:13:02.114424022Z INFO 07-23 03:13:02 metrics.py:345] Avg prompt throughput: 1355.9 tokens/s, Avg generation throughput: 7.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:07.138412366Z INFO 07-23 03:13:07 metrics.py:345] Avg prompt throughput: 1180.3 tokens/s, Avg generation throughput: 2.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:10.703528957Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:10.723774197Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:10.959621972Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:13:12.777150696Z INFO 07-23 03:13:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 14.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:18.311467651Z INFO 07-23 03:13:18 metrics.py:345] Avg prompt throughput: 2960.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:24.528764448Z INFO 07-23 03:13:24 metrics.py:345] Avg prompt throughput: 1976.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:31.514201171Z INFO 07-23 03:13:31 metrics.py:345] Avg prompt throughput: 1759.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:42.028574197Z INFO 07-23 03:13:42 metrics.py:345] Avg prompt throughput: 277.9 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:45.431039577Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:45.437698822Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:45.444970744Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:13:45.625709408Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:13:47.511514890Z INFO 07-23 03:13:47 metrics.py:345] Avg prompt throughput: 747.0 tokens/s, Avg generation throughput: 12.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:52.776792073Z INFO 07-23 03:13:52 metrics.py:345] Avg prompt throughput: 777.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:13:57.800309946Z INFO 07-23 03:13:57 metrics.py:345] Avg prompt throughput: 690.0 tokens/s, Avg generation throughput: 21.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:02.809048544Z INFO 07-23 03:14:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:06.461208755Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:14:06.468193202Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:14:06.479204061Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:14:06.490777942Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:14:06.648810696Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:14:07.835638500Z INFO 07-23 03:14:07 metrics.py:345] Avg prompt throughput: 24.3 tokens/s, Avg generation throughput: 20.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:12.846371069Z INFO 07-23 03:14:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:17.850632021Z INFO 07-23 03:14:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:22.856280984Z INFO 07-23 03:14:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:27.888165813Z INFO 07-23 03:14:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:32.925170980Z INFO 07-23 03:14:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:37.936372523Z INFO 07-23 03:14:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:42.944534615Z INFO 07-23 03:14:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:47.945186825Z INFO 07-23 03:14:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:52.949910632Z INFO 07-23 03:14:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:14:57.964030722Z INFO 07-23 03:14:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:02.982507987Z INFO 07-23 03:15:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:07.998544641Z INFO 07-23 03:15:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:13.004333260Z INFO 07-23 03:15:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:18.016318554Z INFO 07-23 03:15:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:23.029382219Z INFO 07-23 03:15:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:28.041986622Z INFO 07-23 03:15:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:33.065708355Z INFO 07-23 03:15:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:38.078323753Z INFO 07-23 03:15:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:43.091103774Z INFO 07-23 03:15:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:48.108899253Z INFO 07-23 03:15:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:53.135039995Z INFO 07-23 03:15:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:15:58.165410734Z INFO 07-23 03:15:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:03.183514389Z INFO 07-23 03:16:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:08.205135732Z INFO 07-23 03:16:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:13.216887456Z INFO 07-23 03:16:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:18.224956585Z INFO 07-23 03:16:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:20.572927902Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:16:20.746589986Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:16:23.267047992Z INFO 07-23 03:16:23 metrics.py:345] Avg prompt throughput: 107.7 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:28.282762875Z INFO 07-23 03:16:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:33.296712243Z INFO 07-23 03:16:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:38.326019630Z INFO 07-23 03:16:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:43.357754792Z INFO 07-23 03:16:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:48.372927578Z INFO 07-23 03:16:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:53.390454474Z INFO 07-23 03:16:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:16:58.399988303Z INFO 07-23 03:16:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:03.405932485Z INFO 07-23 03:17:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:08.446484958Z INFO 07-23 03:17:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:13.485751050Z INFO 07-23 03:17:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:18.523721424Z INFO 07-23 03:17:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:23.560268144Z INFO 07-23 03:17:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:28.594709480Z INFO 07-23 03:17:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:33.630996999Z INFO 07-23 03:17:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:38.670563497Z INFO 07-23 03:17:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:43.672950199Z INFO 07-23 03:17:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:48.687444828Z INFO 07-23 03:17:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:53.687871665Z INFO 07-23 03:17:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:17:58.699681518Z INFO 07-23 03:17:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:03.710114472Z INFO 07-23 03:18:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:08.720217881Z INFO 07-23 03:18:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:13.743844160Z INFO 07-23 03:18:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:18.764123061Z INFO 07-23 03:18:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:23.778691605Z INFO 07-23 03:18:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:28.790043878Z INFO 07-23 03:18:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:33.801253262Z INFO 07-23 03:18:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:38.807935708Z INFO 07-23 03:18:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:43.815867459Z INFO 07-23 03:18:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:48.827006448Z INFO 07-23 03:18:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:53.834181643Z INFO 07-23 03:18:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:18:58.865607506Z INFO 07-23 03:18:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:03.879818981Z INFO 07-23 03:19:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:08.900000692Z INFO 07-23 03:19:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:10.466463776Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:19:10.478993493Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:19:10.679321643Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:19:14.309575673Z INFO 07-23 03:19:14 metrics.py:345] Avg prompt throughput: 1514.4 tokens/s, Avg generation throughput: 6.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:19.322028134Z INFO 07-23 03:19:19 metrics.py:345] Avg prompt throughput: 972.8 tokens/s, Avg generation throughput: 8.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:24.355050263Z INFO 07-23 03:19:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:27.145073359Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:19:31.802504328Z INFO 07-23 03:19:31 metrics.py:345] Avg prompt throughput: 1008.4 tokens/s, Avg generation throughput: 8.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:36.308014528Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:19:36.316129768Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:19:36.507397695Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:19:38.394733836Z INFO 07-23 03:19:38 metrics.py:345] Avg prompt throughput: 621.3 tokens/s, Avg generation throughput: 15.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:43.661430418Z INFO 07-23 03:19:43 metrics.py:345] Avg prompt throughput: 1761.8 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:48.702576396Z INFO 07-23 03:19:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:53.735280536Z INFO 07-23 03:19:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:19:55.502354844Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:20:00.322746405Z INFO 07-23 03:20:00 metrics.py:345] Avg prompt throughput: 1168.0 tokens/s, Avg generation throughput: 5.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:02.888919108Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:20:02.897148113Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:20:03.074751698Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:20:06.698577320Z INFO 07-23 03:20:06 metrics.py:345] Avg prompt throughput: 1284.8 tokens/s, Avg generation throughput: 9.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:11.953751610Z INFO 07-23 03:20:11 metrics.py:345] Avg prompt throughput: 1225.3 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:16.958423659Z INFO 07-23 03:20:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:21.958724991Z INFO 07-23 03:20:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:26.962080264Z INFO 07-23 03:20:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:31.188353870Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:20:31.346968400Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:20:31.986096383Z INFO 07-23 03:20:31 metrics.py:345] Avg prompt throughput: 65.5 tokens/s, Avg generation throughput: 19.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:36.991956897Z INFO 07-23 03:20:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:42.022761244Z INFO 07-23 03:20:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:47.065410115Z INFO 07-23 03:20:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:52.071533963Z INFO 07-23 03:20:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:20:57.095045358Z INFO 07-23 03:20:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:02.100354460Z INFO 07-23 03:21:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:07.120462554Z INFO 07-23 03:21:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:12.144278143Z INFO 07-23 03:21:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:17.160746225Z INFO 07-23 03:21:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:22.183849444Z INFO 07-23 03:21:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:27.201981662Z INFO 07-23 03:21:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:32.222207283Z INFO 07-23 03:21:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:37.226790110Z INFO 07-23 03:21:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:42.263162787Z INFO 07-23 03:21:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:47.285217965Z INFO 07-23 03:21:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:52.294860803Z INFO 07-23 03:21:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:21:57.305607847Z INFO 07-23 03:21:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:02.332677933Z INFO 07-23 03:22:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:07.361450993Z INFO 07-23 03:22:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:12.386346091Z INFO 07-23 03:22:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:17.425448234Z INFO 07-23 03:22:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:22.445119521Z INFO 07-23 03:22:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:27.467709607Z INFO 07-23 03:22:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:32.488520263Z INFO 07-23 03:22:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:37.510941058Z INFO 07-23 03:22:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:42.548234171Z INFO 07-23 03:22:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:47.581833368Z INFO 07-23 03:22:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:52.623687606Z INFO 07-23 03:22:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:22:57.640445190Z INFO 07-23 03:22:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:02.662738298Z INFO 07-23 03:23:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:04.568763829Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:23:04.740545928Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:23:07.680005320Z INFO 07-23 03:23:07 metrics.py:345] Avg prompt throughput: 228.6 tokens/s, Avg generation throughput: 18.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:12.697581987Z INFO 07-23 03:23:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:17.717154268Z INFO 07-23 03:23:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:22.737304950Z INFO 07-23 03:23:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:27.757062716Z INFO 07-23 03:23:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:32.761707602Z INFO 07-23 03:23:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:37.779115679Z INFO 07-23 03:23:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:42.787757479Z INFO 07-23 03:23:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:47.821253928Z INFO 07-23 03:23:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:52.843645340Z INFO 07-23 03:23:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:23:57.888462280Z INFO 07-23 03:23:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:02.888897990Z INFO 07-23 03:24:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:07.905867858Z INFO 07-23 03:24:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:12.928719262Z INFO 07-23 03:24:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:17.947424275Z INFO 07-23 03:24:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:22.973047988Z INFO 07-23 03:24:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:27.993404362Z INFO 07-23 03:24:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:33.038567398Z INFO 07-23 03:24:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:38.072799979Z INFO 07-23 03:24:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:43.107134360Z INFO 07-23 03:24:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:48.139340196Z INFO 07-23 03:24:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:53.158864324Z INFO 07-23 03:24:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:24:58.184946563Z INFO 07-23 03:24:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:03.205227792Z INFO 07-23 03:25:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:08.227341986Z INFO 07-23 03:25:08 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:13.266499819Z INFO 07-23 03:25:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:18.273249978Z INFO 07-23 03:25:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:23.310387392Z INFO 07-23 03:25:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:28.347385529Z INFO 07-23 03:25:28 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:33.372635744Z INFO 07-23 03:25:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:38.389145564Z INFO 07-23 03:25:38 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:43.404210116Z INFO 07-23 03:25:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:48.431958800Z INFO 07-23 03:25:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:53.458237948Z INFO 07-23 03:25:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:25:58.483877167Z INFO 07-23 03:25:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:03.522336181Z INFO 07-23 03:26:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 1.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:05.090021844Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:05.095951361Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:05.295502128Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:26:08.926311950Z INFO 07-23 03:26:08 metrics.py:345] Avg prompt throughput: 1516.0 tokens/s, Avg generation throughput: 6.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:14.842702111Z INFO 07-23 03:26:14 metrics.py:345] Avg prompt throughput: 1164.9 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:19.852487057Z INFO 07-23 03:26:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:20.202580536Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:20.210313004Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:20.384973960Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:26:24.999260358Z INFO 07-23 03:26:24 metrics.py:345] Avg prompt throughput: 1444.2 tokens/s, Avg generation throughput: 1.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:30.041607883Z INFO 07-23 03:26:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:35.048754058Z INFO 07-23 03:26:35 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:39.100631558Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:39.109347200Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:26:39.304823096Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:26:41.188117195Z INFO 07-23 03:26:41 metrics.py:345] Avg prompt throughput: 667.2 tokens/s, Avg generation throughput: 15.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:50.433459245Z INFO 07-23 03:26:50 metrics.py:345] Avg prompt throughput: 1319.3 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:26:55.457907052Z INFO 07-23 03:26:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:00.476478203Z INFO 07-23 03:27:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:01.588214832Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:27:07.025171015Z INFO 07-23 03:27:07 metrics.py:345] Avg prompt throughput: 1876.4 tokens/s, Avg generation throughput: 3.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:12.629720324Z INFO 07-23 03:27:12 metrics.py:345] Avg prompt throughput: 1105.2 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:17.668027044Z INFO 07-23 03:27:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:22.673085829Z INFO 07-23 03:27:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:27.691946553Z INFO 07-23 03:27:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 21.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 6.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:29.151414399Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:27:29.743848827Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:27:29.763701013Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:27:29.917975218Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:27:32.793393663Z INFO 07-23 03:27:32 metrics.py:345] Avg prompt throughput: 802.9 tokens/s, Avg generation throughput: 6.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:37.836214318Z INFO 07-23 03:27:37 metrics.py:345] Avg prompt throughput: 708.7 tokens/s, Avg generation throughput: 14.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:42.859098255Z INFO 07-23 03:27:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:44.256741458Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:27:47.875184669Z INFO 07-23 03:27:47 metrics.py:345] Avg prompt throughput: 1633.2 tokens/s, Avg generation throughput: 5.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:52.899691834Z INFO 07-23 03:27:52 metrics.py:345] Avg prompt throughput: 155.6 tokens/s, Avg generation throughput: 18.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:27:57.919289532Z INFO 07-23 03:27:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:02.702062753Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:28:02.873611630Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:28:03.112144515Z INFO 07-23 03:28:03 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.6 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:08.150500129Z INFO 07-23 03:28:08 metrics.py:345] Avg prompt throughput: 1578.7 tokens/s, Avg generation throughput: 1.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:13.187908126Z INFO 07-23 03:28:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:17.164320870Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:28:17.327688086Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:28:19.210550904Z INFO 07-23 03:28:19 metrics.py:345] Avg prompt throughput: 680.1 tokens/s, Avg generation throughput: 14.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:24.225292173Z INFO 07-23 03:28:24 metrics.py:345] Avg prompt throughput: 800.0 tokens/s, Avg generation throughput: 8.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:29.226764995Z INFO 07-23 03:28:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:31.869759250Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:28:34.263598007Z INFO 07-23 03:28:34 metrics.py:345] Avg prompt throughput: 132.0 tokens/s, Avg generation throughput: 18.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:39.292111567Z INFO 07-23 03:28:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:44.324452648Z INFO 07-23 03:28:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:49.348232646Z INFO 07-23 03:28:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:54.355931285Z INFO 07-23 03:28:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:28:59.363795090Z INFO 07-23 03:28:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 23.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:04.392516862Z INFO 07-23 03:29:04 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:09.415443309Z INFO 07-23 03:29:09 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:14.436739524Z INFO 07-23 03:29:14 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:19.457395216Z INFO 07-23 03:29:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:24.474718167Z INFO 07-23 03:29:24 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:29.501523520Z INFO 07-23 03:29:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:34.523233969Z INFO 07-23 03:29:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:39.552660459Z INFO 07-23 03:29:39 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:44.570080552Z INFO 07-23 03:29:44 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:49.596454253Z INFO 07-23 03:29:49 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:54.615672916Z INFO 07-23 03:29:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:29:59.633932098Z INFO 07-23 03:29:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:02.589700120Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:02.778803816Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:04.677928907Z INFO 07-23 03:30:04 metrics.py:345] Avg prompt throughput: 812.1 tokens/s, Avg generation throughput: 13.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:09.720978113Z INFO 07-23 03:30:09 metrics.py:345] Avg prompt throughput: 952.0 tokens/s, Avg generation throughput: 10.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:11.253121488Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:14.873610618Z INFO 07-23 03:30:14 metrics.py:345] Avg prompt throughput: 1589.9 tokens/s, Avg generation throughput: 5.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:21.905152531Z INFO 07-23 03:30:21 metrics.py:345] Avg prompt throughput: 1420.5 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:26.927246350Z INFO 07-23 03:30:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:26.949212854Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:26.957860631Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:26.966418697Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:27.141252545Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:31.939042290Z INFO 07-23 03:30:31 metrics.py:345] Avg prompt throughput: 1712.4 tokens/s, Avg generation throughput: 2.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:36.949470944Z INFO 07-23 03:30:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:40.533602444Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:40.691497271Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:41.010347463Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:41.018689226Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:41.178014687Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:43.067206037Z INFO 07-23 03:30:43 metrics.py:345] Avg prompt throughput: 685.9 tokens/s, Avg generation throughput: 13.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:48.085902193Z INFO 07-23 03:30:48 metrics.py:345] Avg prompt throughput: 617.9 tokens/s, Avg generation throughput: 11.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:53.113753547Z INFO 07-23 03:30:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:30:53.864374472Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:53.875008951Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:30:54.049288150Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:30:58.801164893Z INFO 07-23 03:30:58 metrics.py:345] Avg prompt throughput: 1346.1 tokens/s, Avg generation throughput: 3.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:01.135308567Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:31:06.029877137Z INFO 07-23 03:31:06 metrics.py:345] Avg prompt throughput: 1074.5 tokens/s, Avg generation throughput: 6.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:08.592483770Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:31:08.960823866Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:31:12.640101178Z INFO 07-23 03:31:12 metrics.py:345] Avg prompt throughput: 1239.3 tokens/s, Avg generation throughput: 8.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:18.383940479Z INFO 07-23 03:31:18 metrics.py:345] Avg prompt throughput: 2139.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:24.879158343Z INFO 07-23 03:31:24 metrics.py:345] Avg prompt throughput: 1891.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:32.155291335Z INFO 07-23 03:31:32 metrics.py:345] Avg prompt throughput: 1688.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:37.443808749Z INFO 07-23 03:31:37 metrics.py:345] Avg prompt throughput: 1549.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:43.111724266Z INFO 07-23 03:31:43 metrics.py:345] Avg prompt throughput: 1445.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:49.164283827Z INFO 07-23 03:31:49 metrics.py:345] Avg prompt throughput: 1353.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:31:55.618757782Z INFO 07-23 03:31:55 metrics.py:345] Avg prompt throughput: 1269.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:02.450252476Z INFO 07-23 03:32:02 metrics.py:345] Avg prompt throughput: 1199.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:09.683638726Z INFO 07-23 03:32:09 metrics.py:345] Avg prompt throughput: 1132.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:17.325293296Z INFO 07-23 03:32:17 metrics.py:345] Avg prompt throughput: 1072.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:23.119540943Z INFO 07-23 03:32:23 metrics.py:345] Avg prompt throughput: 706.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:33.119590916Z INFO 07-23 03:32:33 metrics.py:345] Avg prompt throughput: 409.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:43.120993751Z INFO 07-23 03:32:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:51.139668086Z INFO 07-23 03:32:51 metrics.py:345] Avg prompt throughput: 349.9 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:32:56.182102278Z INFO 07-23 03:32:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:01.222431844Z INFO 07-23 03:33:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:06.255444585Z INFO 07-23 03:33:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:11.294163817Z INFO 07-23 03:33:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 36.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:16.330648490Z INFO 07-23 03:33:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:21.366805857Z INFO 07-23 03:33:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:26.401669203Z INFO 07-23 03:33:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:31.427649231Z INFO 07-23 03:33:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:36.462316305Z INFO 07-23 03:33:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:41.488932088Z INFO 07-23 03:33:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:46.518394807Z INFO 07-23 03:33:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:51.547857990Z INFO 07-23 03:33:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:33:56.580255579Z INFO 07-23 03:33:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:01.580987774Z INFO 07-23 03:34:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:06.616073565Z INFO 07-23 03:34:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:11.650597177Z INFO 07-23 03:34:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:16.654305379Z INFO 07-23 03:34:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:21.683189679Z INFO 07-23 03:34:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:26.707783194Z INFO 07-23 03:34:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:31.739851567Z INFO 07-23 03:34:31 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:36.764364675Z INFO 07-23 03:34:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:41.785282509Z INFO 07-23 03:34:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:46.807385509Z INFO 07-23 03:34:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:51.832113797Z INFO 07-23 03:34:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:34:56.852758637Z INFO 07-23 03:34:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:01.877195030Z INFO 07-23 03:35:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:06.909124996Z INFO 07-23 03:35:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:11.936863441Z INFO 07-23 03:35:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:16.969294745Z INFO 07-23 03:35:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:22.001685150Z INFO 07-23 03:35:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:27.035357579Z INFO 07-23 03:35:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:32.070780441Z INFO 07-23 03:35:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:37.109800889Z INFO 07-23 03:35:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:42.144817317Z INFO 07-23 03:35:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:47.182019980Z INFO 07-23 03:35:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 37.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:52.225007586Z INFO 07-23 03:35:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:35:57.259046506Z INFO 07-23 03:35:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:02.297575559Z INFO 07-23 03:36:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:07.340493043Z INFO 07-23 03:36:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:12.378520548Z INFO 07-23 03:36:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:17.409447662Z INFO 07-23 03:36:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:22.439119517Z INFO 07-23 03:36:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:27.442704506Z INFO 07-23 03:36:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:32.480948868Z INFO 07-23 03:36:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:37.529398347Z INFO 07-23 03:36:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:42.574099184Z INFO 07-23 03:36:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:47.612483559Z INFO 07-23 03:36:47 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:52.652951855Z INFO 07-23 03:36:52 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:36:57.687904692Z INFO 07-23 03:36:57 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:02.726593377Z INFO 07-23 03:37:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:07.730913728Z INFO 07-23 03:37:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:12.774245347Z INFO 07-23 03:37:12 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:17.813872698Z INFO 07-23 03:37:17 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:22.848066921Z INFO 07-23 03:37:22 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:27.882473147Z INFO 07-23 03:37:27 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:32.917401272Z INFO 07-23 03:37:32 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:37.932243776Z INFO 07-23 03:37:37 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:42.963779527Z INFO 07-23 03:37:42 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:48.007317544Z INFO 07-23 03:37:48 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:53.048938883Z INFO 07-23 03:37:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:58.085090714Z INFO 07-23 03:37:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 38.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:37:59.337134045Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:03.126573524Z INFO 07-23 03:38:03 metrics.py:345] Avg prompt throughput: 1624.9 tokens/s, Avg generation throughput: 4.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:07.915203230Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:38:07.925057068Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:38:08.094963074Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:09.982564809Z INFO 07-23 03:38:09 metrics.py:345] Avg prompt throughput: 715.7 tokens/s, Avg generation throughput: 12.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:14.987905774Z INFO 07-23 03:38:14 metrics.py:345] Avg prompt throughput: 1116.8 tokens/s, Avg generation throughput: 7.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:19.993703211Z INFO 07-23 03:38:19 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:23.734983446Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:25.623769661Z INFO 07-23 03:38:25 metrics.py:345] Avg prompt throughput: 727.5 tokens/s, Avg generation throughput: 14.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:30.635619226Z INFO 07-23 03:38:30 metrics.py:345] Avg prompt throughput: 1021.6 tokens/s, Avg generation throughput: 12.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:34.343758361Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:38:34.566766277Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:36.464600680Z INFO 07-23 03:38:36 metrics.py:345] Avg prompt throughput: 702.7 tokens/s, Avg generation throughput: 14.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:41.913518197Z INFO 07-23 03:38:41 metrics.py:345] Avg prompt throughput: 2255.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:53.127802320Z INFO 07-23 03:38:53 metrics.py:345] Avg prompt throughput: 730.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:38:57.626063560Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:38:59.523823531Z INFO 07-23 03:38:59 metrics.py:345] Avg prompt throughput: 1128.4 tokens/s, Avg generation throughput: 14.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:04.973947484Z INFO 07-23 03:39:04 metrics.py:345] Avg prompt throughput: 2254.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:13.127817976Z INFO 07-23 03:39:13 metrics.py:345] Avg prompt throughput: 502.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:18.136657170Z INFO 07-23 03:39:18 metrics.py:345] Avg prompt throughput: 697.6 tokens/s, Avg generation throughput: 16.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:19.550416132Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:39:23.976378415Z INFO 07-23 03:39:23 metrics.py:345] Avg prompt throughput: 1237.9 tokens/s, Avg generation throughput: 4.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:29.002950812Z INFO 07-23 03:39:29 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:34.041860533Z INFO 07-23 03:39:34 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:34.620637687Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:39:41.578722845Z INFO 07-23 03:39:41 metrics.py:345] Avg prompt throughput: 1490.8 tokens/s, Avg generation throughput: 1.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:46.612817940Z INFO 07-23 03:39:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:51.651629828Z INFO 07-23 03:39:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:56.654212752Z INFO 07-23 03:39:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:39:58.976469155Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:40:02.603723546Z INFO 07-23 03:40:02 metrics.py:345] Avg prompt throughput: 1376.9 tokens/s, Avg generation throughput: 7.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:08.305651965Z INFO 07-23 03:40:08 metrics.py:345] Avg prompt throughput: 2155.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:16.236480761Z INFO 07-23 03:40:16 metrics.py:345] Avg prompt throughput: 478.5 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:21.267816676Z INFO 07-23 03:40:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 7.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:26.295602231Z INFO 07-23 03:40:26 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 8.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:28.778018078Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:40:32.408324583Z INFO 07-23 03:40:32 metrics.py:345] Avg prompt throughput: 1340.2 tokens/s, Avg generation throughput: 8.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:38.108997575Z INFO 07-23 03:40:38 metrics.py:345] Avg prompt throughput: 2155.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:43.129334599Z INFO 07-23 03:40:43 metrics.py:345] Avg prompt throughput: 1631.7 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:48.160479129Z INFO 07-23 03:40:48 metrics.py:345] Avg prompt throughput: 120.6 tokens/s, Avg generation throughput: 16.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:53.164874777Z INFO 07-23 03:40:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:58.169636135Z INFO 07-23 03:40:58 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:40:59.795739240Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:41:03.414206853Z INFO 07-23 03:41:03 metrics.py:345] Avg prompt throughput: 1562.0 tokens/s, Avg generation throughput: 5.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:08.455624617Z INFO 07-23 03:41:08 metrics.py:345] Avg prompt throughput: 728.7 tokens/s, Avg generation throughput: 4.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:13.456923903Z INFO 07-23 03:41:13 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:18.485525514Z INFO 07-23 03:41:18 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:23.498236924Z INFO 07-23 03:41:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:26.268303762Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:41:31.070539610Z INFO 07-23 03:41:31 metrics.py:345] Avg prompt throughput: 1015.4 tokens/s, Avg generation throughput: 7.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:36.096941176Z INFO 07-23 03:41:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:41.130726930Z INFO 07-23 03:41:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 2.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:41.559287741Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:41:41.969406342Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:41:47.474648078Z INFO 07-23 03:41:47 metrics.py:345] Avg prompt throughput: 1937.0 tokens/s, Avg generation throughput: 1.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:41:53.131561420Z INFO 07-23 03:41:53 metrics.py:345] Avg prompt throughput: 1448.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:00.176943964Z INFO 07-23 03:42:00 metrics.py:345] Avg prompt throughput: 2325.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:07.700147555Z INFO 07-23 03:42:07 metrics.py:345] Avg prompt throughput: 1633.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:13.132480415Z INFO 07-23 03:42:13 metrics.py:345] Avg prompt throughput: 754.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:19.035572074Z INFO 07-23 03:42:19 metrics.py:345] Avg prompt throughput: 2081.6 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:33.133304474Z INFO 07-23 03:42:33 metrics.py:345] Avg prompt throughput: 290.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:43.134438022Z INFO 07-23 03:42:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:48.154183452Z INFO 07-23 03:42:48 metrics.py:345] Avg prompt throughput: 735.3 tokens/s, Avg generation throughput: 17.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:53.191208919Z INFO 07-23 03:42:53 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 23.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:42:57.644819022Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:42:59.536790168Z INFO 07-23 03:42:59 metrics.py:345] Avg prompt throughput: 645.5 tokens/s, Avg generation throughput: 13.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:04.984151481Z INFO 07-23 03:43:04 metrics.py:345] Avg prompt throughput: 2255.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:13.134871432Z INFO 07-23 03:43:13 metrics.py:345] Avg prompt throughput: 1005.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:18.371162432Z INFO 07-23 03:43:18 metrics.py:345] Avg prompt throughput: 757.4 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:23.343108547Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:43:23.588052243Z INFO 07-23 03:43:23 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.6 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:28.848071558Z INFO 07-23 03:43:28 metrics.py:345] Avg prompt throughput: 2336.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:34.822293294Z INFO 07-23 03:43:34 metrics.py:345] Avg prompt throughput: 2056.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:41.559053064Z INFO 07-23 03:43:41 metrics.py:345] Avg prompt throughput: 1824.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:42.134554199Z INFO:     172.28.22.23:54024 - "GET /v1/models HTTP/1.1" 200 OK
+2026-07-23T03:43:49.093692247Z INFO 07-23 03:43:49 metrics.py:345] Avg prompt throughput: 1630.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:43:54.573982630Z INFO 07-23 03:43:54 metrics.py:345] Avg prompt throughput: 1494.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:00.419430557Z INFO 07-23 03:44:00 metrics.py:345] Avg prompt throughput: 1401.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:06.652667894Z INFO 07-23 03:44:06 metrics.py:345] Avg prompt throughput: 1314.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:13.273466906Z INFO 07-23 03:44:13 metrics.py:345] Avg prompt throughput: 1237.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:23.589944720Z INFO 07-23 03:44:23 metrics.py:345] Avg prompt throughput: 397.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:33.589622516Z INFO 07-23 03:44:33 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:40.373070879Z INFO 07-23 03:44:40 metrics.py:345] Avg prompt throughput: 477.0 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:45.378779262Z INFO 07-23 03:44:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 29.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:49.409870456Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:44:51.294134680Z INFO 07-23 03:44:51 metrics.py:345] Avg prompt throughput: 692.4 tokens/s, Avg generation throughput: 13.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:44:57.327566458Z INFO 07-23 03:44:57 metrics.py:345] Avg prompt throughput: 1304.6 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:02.343004242Z INFO 07-23 03:45:02 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 3.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:07.353037375Z INFO 07-23 03:45:07 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:08.703038876Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:45:08.945392988Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:45:12.589865901Z INFO 07-23 03:45:12 metrics.py:345] Avg prompt throughput: 1564.3 tokens/s, Avg generation throughput: 5.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:18.308064429Z INFO 07-23 03:45:18 metrics.py:345] Avg prompt throughput: 2148.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:23.590532919Z INFO 07-23 03:45:23 metrics.py:345] Avg prompt throughput: 1550.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:29.508225758Z INFO 07-23 03:45:29 metrics.py:345] Avg prompt throughput: 2076.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:40.164335600Z INFO 07-23 03:45:40 metrics.py:345] Avg prompt throughput: 286.5 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:45.215691270Z INFO 07-23 03:45:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:50.262555523Z INFO 07-23 03:45:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:45:55.305684193Z INFO 07-23 03:45:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:00.329990814Z INFO 07-23 03:46:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:05.358893348Z INFO 07-23 03:46:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:10.388523949Z INFO 07-23 03:46:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:15.420504664Z INFO 07-23 03:46:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:20.448707580Z INFO 07-23 03:46:20 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:25.464481774Z INFO 07-23 03:46:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:30.502872201Z INFO 07-23 03:46:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:35.530078168Z INFO 07-23 03:46:35 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:40.559825975Z INFO 07-23 03:46:40 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:45.595368097Z INFO 07-23 03:46:45 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:50.632083164Z INFO 07-23 03:46:50 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:46:55.670123461Z INFO 07-23 03:46:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.7 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:00.692064975Z INFO 07-23 03:47:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:05.731449900Z INFO 07-23 03:47:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:10.775862113Z INFO 07-23 03:47:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:15.789807502Z INFO 07-23 03:47:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 14.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:18.030344386Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:47:18.267069376Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:47:21.908553681Z INFO 07-23 03:47:21 metrics.py:345] Avg prompt throughput: 1338.8 tokens/s, Avg generation throughput: 7.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:27.607452205Z INFO 07-23 03:47:27 metrics.py:345] Avg prompt throughput: 2156.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:33.592167279Z INFO 07-23 03:47:33 metrics.py:345] Avg prompt throughput: 684.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:38.615627195Z INFO 07-23 03:47:38 metrics.py:345] Avg prompt throughput: 610.3 tokens/s, Avg generation throughput: 8.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:43.652542974Z INFO 07-23 03:47:43 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:48.069171000Z ERROR 07-23 03:47:48 serving_chat.py:221] Error in applying chat template from request
+2026-07-23T03:47:48.069190665Z ERROR 07-23 03:47:48 serving_chat.py:221] Traceback (most recent call last):
+2026-07-23T03:47:48.069193533Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 188, in create_chat_completion
+2026-07-23T03:47:48.069196058Z ERROR 07-23 03:47:48 serving_chat.py:221]     conversation, mm_data_future = parse_chat_messages_futures(
+2026-07-23T03:47:48.069198382Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 551, in parse_chat_messages_futures
+2026-07-23T03:47:48.069200645Z ERROR 07-23 03:47:48 serving_chat.py:221]     sub_messages = _parse_chat_message_content(msg, mm_tracker)
+2026-07-23T03:47:48.069202837Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 477, in _parse_chat_message_content
+2026-07-23T03:47:48.069204889Z ERROR 07-23 03:47:48 serving_chat.py:221]     result = _parse_chat_message_content_parts(
+2026-07-23T03:47:48.069206940Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 429, in _parse_chat_message_content_parts
+2026-07-23T03:47:48.069209541Z ERROR 07-23 03:47:48 serving_chat.py:221]     mm_parser.parse_image(image_url["url"])
+2026-07-23T03:47:48.069211572Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 311, in parse_image
+2026-07-23T03:47:48.069213819Z ERROR 07-23 03:47:48 serving_chat.py:221]     placeholder = self._tracker.add("image", image_coro)
+2026-07-23T03:47:48.069215830Z ERROR 07-23 03:47:48 serving_chat.py:221]   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/chat_utils.py", line 220, in add
+2026-07-23T03:47:48.069218119Z ERROR 07-23 03:47:48 serving_chat.py:221]     raise ValueError(
+2026-07-23T03:47:48.069229290Z ERROR 07-23 03:47:48 serving_chat.py:221] ValueError: At most 1 image(s) may be provided in one request.
+2026-07-23T03:47:48.069293988Z /usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py:222: RuntimeWarning: coroutine 'async_get_and_parse_image' was never awaited
+2026-07-23T03:47:48.069300137Z   return self.create_error_response(str(e))
+2026-07-23T03:47:48.069302341Z RuntimeWarning: Enable tracemalloc to get the object allocation traceback
+2026-07-23T03:47:48.069481495Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:47:48.325722518Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:47:50.228821656Z INFO 07-23 03:47:50 metrics.py:345] Avg prompt throughput: 622.9 tokens/s, Avg generation throughput: 14.8 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:47:55.685485411Z INFO 07-23 03:47:55 metrics.py:345] Avg prompt throughput: 2251.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:01.889056918Z INFO 07-23 03:48:01 metrics.py:345] Avg prompt throughput: 1980.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:08.880652390Z INFO 07-23 03:48:08 metrics.py:345] Avg prompt throughput: 1757.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:20.950602402Z INFO 07-23 03:48:20 metrics.py:345] Avg prompt throughput: 547.1 tokens/s, Avg generation throughput: 0.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:25.952959584Z INFO 07-23 03:48:25 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:30.999396546Z INFO 07-23 03:48:30 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:36.044256010Z INFO 07-23 03:48:36 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:41.044599828Z INFO 07-23 03:48:41 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:46.083073696Z INFO 07-23 03:48:46 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:51.124521665Z INFO 07-23 03:48:51 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.6%, CPU KV cache usage: 0.0%.
+2026-07-23T03:48:56.168277560Z INFO 07-23 03:48:56 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:01.212516850Z INFO 07-23 03:49:01 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:06.258871180Z INFO 07-23 03:49:06 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.7%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:11.303607987Z INFO 07-23 03:49:11 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:16.352861160Z INFO 07-23 03:49:16 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:21.355919111Z INFO 07-23 03:49:21 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 15.8%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:23.979457774Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:49:27.605136815Z INFO 07-23 03:49:27 metrics.py:345] Avg prompt throughput: 1310.9 tokens/s, Avg generation throughput: 7.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:33.296245524Z INFO 07-23 03:49:33 metrics.py:345] Avg prompt throughput: 2159.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:38.656394179Z INFO 07-23 03:49:38 metrics.py:345] Avg prompt throughput: 1597.3 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.4%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:41.259500869Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:49:44.876448503Z INFO 07-23 03:49:44 metrics.py:345] Avg prompt throughput: 1317.1 tokens/s, Avg generation throughput: 8.5 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:50.779280791Z INFO 07-23 03:49:50 metrics.py:345] Avg prompt throughput: 1163.2 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:49:55.782459543Z INFO 07-23 03:49:55 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 4.9%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:00.821850285Z INFO 07-23 03:50:00 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:05.860302968Z INFO 07-23 03:50:05 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:10.896836932Z INFO 07-23 03:50:10 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.0%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:15.907192499Z INFO 07-23 03:50:15 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.1%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:19.131493471Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:50:21.026655055Z INFO 07-23 03:50:21 metrics.py:345] Avg prompt throughput: 800.1 tokens/s, Avg generation throughput: 13.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:26.482545766Z INFO 07-23 03:50:26 metrics.py:345] Avg prompt throughput: 2252.2 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:32.675695952Z INFO 07-23 03:50:32 metrics.py:345] Avg prompt throughput: 1984.1 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:37.717894719Z INFO 07-23 03:50:37 metrics.py:345] Avg prompt throughput: 104.5 tokens/s, Avg generation throughput: 13.9 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 9.5%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:38.747799235Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:50:43.596272691Z INFO 07-23 03:50:43 metrics.py:345] Avg prompt throughput: 1393.6 tokens/s, Avg generation throughput: 3.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:49.494533413Z INFO 07-23 03:50:49 metrics.py:345] Avg prompt throughput: 1317.8 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:54.495926964Z INFO 07-23 03:50:54 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.6 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.2%, CPU KV cache usage: 0.0%.
+2026-07-23T03:50:59.532585217Z INFO 07-23 03:50:59 metrics.py:345] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 22.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 5.3%, CPU KV cache usage: 0.0%.
+2026-07-23T03:51:04.243701758Z INFO:     172.28.22.23:38250 - "POST /v1/chat/completions HTTP/1.1" 200 OK
+2026-07-23T03:51:04.695557111Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] Exception in worker VllmWorkerProcess while processing method start_worker_execution_loop: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 2 has a total capacty of 31.72 GiB of which 70.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF, Traceback (most recent call last):
+2026-07-23T03:51:04.695590033Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_worker_utils.py", line 224, in _run_worker_process
+2026-07-23T03:51:04.695593398Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = executor(*args, **kwargs)
+2026-07-23T03:51:04.695597304Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.695599822Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
+2026-07-23T03:51:04.695601960Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 69, in start_worker_execution_loop
+2026-07-23T03:51:04.695604422Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = self.execute_model(execute_model_req=None)
+2026-07-23T03:51:04.695606461Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:04.695608857Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = self.model_runner.execute_model(
+2026-07-23T03:51:04.695611175Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.695613359Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
+2026-07-23T03:51:04.695615304Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:04.695617660Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:04.695619640Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695621844Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695623828Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695628609Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695630599Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:04.695632812Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states = self.model(
+2026-07-23T03:51:04.695634801Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695636954Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695639175Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695641326Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695643268Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:04.695645430Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states, residual = layer(
+2026-07-23T03:51:04.695647355Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695649500Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695651500Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695653916Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695655924Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:04.695658142Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:04.695660265Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695662484Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695667765Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695670920Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695672956Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:04.695675139Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:04.695677985Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:04.695680320Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self.forward_native(x, residual)
+2026-07-23T03:51:04.695684136Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:04.695686306Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:04.695688422Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 158, in forward_static
+2026-07-23T03:51:04.695690647Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     variance = x.pow(2).mean(dim=-1, keepdim=True)
+2026-07-23T03:51:04.695693383Z (VllmWorkerProcess pid=345) (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] Exception in worker VllmWorkerProcess while processing method start_worker_execution_loop: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 1 has a total capacty of 31.72 GiB of which 74.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF, Traceback (most recent call last):
+2026-07-23T03:51:04.695697049Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_worker_utils.py", line 224, in _run_worker_process
+2026-07-23T03:51:04.695699616Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = executor(*args, **kwargs)
+2026-07-23T03:51:04.695701640Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.695706279Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
+2026-07-23T03:51:04.695708636Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 69, in start_worker_execution_loop
+2026-07-23T03:51:04.695710796Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = self.execute_model(execute_model_req=None)
+2026-07-23T03:51:04.695712766Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:04.695714957Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     output = self.model_runner.execute_model(
+2026-07-23T03:51:04.695716932Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.695719271Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
+2026-07-23T03:51:04.695721336Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:04.695723498Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:04.695725472Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695727695Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695729775Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695732069Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695734030Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:04.695736157Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states = self.model(
+2026-07-23T03:51:04.695738121Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695742683Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695744681Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695746918Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695748963Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:04.695751447Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states, residual = layer(
+2026-07-23T03:51:04.695753660Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695755775Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695757874Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695759993Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695762002Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:04.695764118Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:04.695766114Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.695768209Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.695770214Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.695772460Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.695774624Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:04.695778783Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:04.695781237Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:04.695783998Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self.forward_native(x, residual)
+2026-07-23T03:51:04.695785906Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:04.695788026Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:04.695790208Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 158, in forward_static
+2026-07-23T03:51:04.695792433Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231]     variance = x.pow(2).mean(dim=-1, keepdim=True)
+2026-07-23T03:51:04.695794820Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 2 has a total capacty of 31.72 GiB of which 70.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:04.695797659Z (VllmWorkerProcess pid=345) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] 
+2026-07-23T03:51:04.695799657Z ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 1 has a total capacty of 31.72 GiB of which 74.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:04.695802893Z (VllmWorkerProcess pid=344) ERROR 07-23 03:51:04 multiproc_worker_utils.py:231] 
+2026-07-23T03:51:04.699701286Z ERROR 07-23 03:51:04 async_llm_engine.py:66] Engine background task failed
+2026-07-23T03:51:04.699707282Z ERROR 07-23 03:51:04 async_llm_engine.py:66] Traceback (most recent call last):
+2026-07-23T03:51:04.699709690Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 56, in _log_task_completion
+2026-07-23T03:51:04.699712553Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return_value = task.result()
+2026-07-23T03:51:04.699722044Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 857, in run_engine_loop
+2026-07-23T03:51:04.699724371Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     result = task.result()
+2026-07-23T03:51:04.699726357Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 776, in engine_step
+2026-07-23T03:51:04.699728437Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     request_outputs = await self.engine.step_async(virtual_engine)
+2026-07-23T03:51:04.699730549Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 348, in step_async
+2026-07-23T03:51:04.699732927Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     outputs = await self.model_executor.execute_model_async(
+2026-07-23T03:51:04.699734814Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/distributed_gpu_executor.py", line 181, in execute_model_async
+2026-07-23T03:51:04.699736873Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return await self._driver_execute_model_async(execute_model_req)
+2026-07-23T03:51:04.699738752Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 224, in _driver_execute_model_async
+2026-07-23T03:51:04.699740932Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return await self.driver_exec_model(execute_model_req)
+2026-07-23T03:51:04.699743070Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
+2026-07-23T03:51:04.699745042Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     result = self.fn(*self.args, **self.kwargs)
+2026-07-23T03:51:04.699747316Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:04.699749564Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     output = self.model_runner.execute_model(
+2026-07-23T03:51:04.699751539Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.699753508Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return func(*args, **kwargs)
+2026-07-23T03:51:04.699755607Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:04.699758036Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:04.699760153Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.699762173Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.699764016Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.699768920Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.699770911Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:04.699772926Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     hidden_states = self.model(
+2026-07-23T03:51:04.699774758Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.699776744Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.699778565Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.699780536Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.699782347Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:04.699784609Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     hidden_states, residual = layer(
+2026-07-23T03:51:04.699786447Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.699788829Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.699790628Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.699792704Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.699856811Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:04.699859662Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:04.699861913Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.699864092Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.699866211Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.699868266Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.699870390Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:04.699872478Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:04.699876920Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:04.699879423Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self.forward_native(x, residual)
+2026-07-23T03:51:04.699881751Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:04.699884537Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:04.699886523Z ERROR 07-23 03:51:04 async_llm_engine.py:66]   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 157, in forward_static
+2026-07-23T03:51:04.699888605Z ERROR 07-23 03:51:04 async_llm_engine.py:66]     x = x.float()
+2026-07-23T03:51:04.699890682Z ERROR 07-23 03:51:04 async_llm_engine.py:66] torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacty of 31.72 GiB of which 50.92 MiB is free. Of the allocated memory 30.86 GiB is allocated by PyTorch, and 210.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:04.700138219Z Exception in callback functools.partial(<function _log_task_completion at 0x7f206fedf1c0>, error_callback=<bound method AsyncLLMEngine._error_callback of <vllm.engine.async_llm_engine.AsyncLLMEngine object at 0x7f206034f010>>)
+2026-07-23T03:51:04.700142915Z handle: <Handle functools.partial(<function _log_task_completion at 0x7f206fedf1c0>, error_callback=<bound method AsyncLLMEngine._error_callback of <vllm.engine.async_llm_engine.AsyncLLMEngine object at 0x7f206034f010>>)>
+2026-07-23T03:51:04.700146529Z Traceback (most recent call last):
+2026-07-23T03:51:04.700148465Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 56, in _log_task_completion
+2026-07-23T03:51:04.700150727Z     return_value = task.result()
+2026-07-23T03:51:04.700152569Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 857, in run_engine_loop
+2026-07-23T03:51:04.700154521Z     result = task.result()
+2026-07-23T03:51:04.700156285Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 776, in engine_step
+2026-07-23T03:51:04.700158380Z     request_outputs = await self.engine.step_async(virtual_engine)
+2026-07-23T03:51:04.700160278Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 348, in step_async
+2026-07-23T03:51:04.700162359Z     outputs = await self.model_executor.execute_model_async(
+2026-07-23T03:51:04.700164171Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/distributed_gpu_executor.py", line 181, in execute_model_async
+2026-07-23T03:51:04.700166399Z     return await self._driver_execute_model_async(execute_model_req)
+2026-07-23T03:51:04.700170872Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 224, in _driver_execute_model_async
+2026-07-23T03:51:04.700172919Z     return await self.driver_exec_model(execute_model_req)
+2026-07-23T03:51:04.700174704Z   File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
+2026-07-23T03:51:04.700176814Z     result = self.fn(*self.args, **self.kwargs)
+2026-07-23T03:51:04.700178633Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:04.700180563Z     output = self.model_runner.execute_model(
+2026-07-23T03:51:04.700182330Z   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:04.700184286Z     return func(*args, **kwargs)
+2026-07-23T03:51:04.700186064Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:04.700188266Z     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:04.700190129Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.700192031Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.700193904Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.700195807Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.700197556Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:04.700199508Z     hidden_states = self.model(
+2026-07-23T03:51:04.700201263Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.700203336Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.700205463Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.700207334Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.700209080Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:04.700211063Z     hidden_states, residual = layer(
+2026-07-23T03:51:04.700212851Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.700214768Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.700217261Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.700219719Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.700221463Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:04.700223405Z     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:04.700228613Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:04.700230566Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:04.700232462Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:04.700234414Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:04.700236143Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:04.700238046Z     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:04.700239806Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:04.700241998Z     return self.forward_native(x, residual)
+2026-07-23T03:51:04.700243823Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:04.700245757Z     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:04.700247627Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 157, in forward_static
+2026-07-23T03:51:04.700249543Z     x = x.float()
+2026-07-23T03:51:04.700252832Z torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacty of 31.72 GiB of which 50.92 MiB is free. Of the allocated memory 30.86 GiB is allocated by PyTorch, and 210.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:04.700256610Z 
+2026-07-23T03:51:04.700258408Z The above exception was the direct cause of the following exception:
+2026-07-23T03:51:04.700260214Z 
+2026-07-23T03:51:04.700261917Z Traceback (most recent call last):
+2026-07-23T03:51:04.700263623Z   File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run
+2026-07-23T03:51:04.700265499Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 68, in _log_task_completion
+2026-07-23T03:51:04.700268134Z     raise AsyncEngineDeadError(
+2026-07-23T03:51:04.700269863Z vllm.engine.async_llm_engine.AsyncEngineDeadError: Task finished unexpectedly. This should never happen! Please open an issue on Github. See stack trace above for the actual cause.
+2026-07-23T03:51:04.702552963Z ERROR:    Exception in ASGI application
+2026-07-23T03:51:04.702558866Z Traceback (most recent call last):
+2026-07-23T03:51:04.702560834Z   File "/usr/local/lib/python3.10/site-packages/starlette/_utils.py", line 79, in collapse_excgroups
+2026-07-23T03:51:04.702564379Z     yield
+2026-07-23T03:51:04.702566507Z   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 271, in __call__
+2026-07-23T03:51:04.702571357Z     async with anyio.create_task_group() as task_group:
+2026-07-23T03:51:04.702573227Z   File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 772, in __aexit__
+2026-07-23T03:51:04.702575469Z     raise BaseExceptionGroup(
+2026-07-23T03:51:04.702577231Z exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
+2026-07-23T03:51:04.702579140Z 
+2026-07-23T03:51:04.702580984Z During handling of the above exception, another exception occurred:
+2026-07-23T03:51:04.702582850Z 
+2026-07-23T03:51:04.702584544Z Traceback (most recent call last):
+2026-07-23T03:51:04.702586535Z   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
+2026-07-23T03:51:04.702588429Z     await app(scope, receive, sender)
+2026-07-23T03:51:04.702590235Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
+2026-07-23T03:51:04.702592118Z     await response(scope, receive, send)
+2026-07-23T03:51:04.702593877Z   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 270, in __call__
+2026-07-23T03:51:04.702595840Z     with collapse_excgroups():
+2026-07-23T03:51:04.702597561Z   File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
+2026-07-23T03:51:04.702599394Z     self.gen.throw(typ, value, traceback)
+2026-07-23T03:51:04.702601159Z   File "/usr/local/lib/python3.10/site-packages/starlette/_utils.py", line 85, in collapse_excgroups
+2026-07-23T03:51:04.702603068Z     raise exc
+2026-07-23T03:51:04.702604874Z   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 274, in wrap
+2026-07-23T03:51:04.702606737Z     await func()
+2026-07-23T03:51:04.702608502Z   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 254, in stream_response
+2026-07-23T03:51:04.702610428Z     async for chunk in self.body_iterator:
+2026-07-23T03:51:04.702612210Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 846, in chat_completion_stream_generator
+2026-07-23T03:51:04.702614205Z     await self.engine_client.abort(request_id)
+2026-07-23T03:51:04.702615921Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 1243, in abort
+2026-07-23T03:51:04.702617838Z     raise AsyncEngineDeadError(
+2026-07-23T03:51:04.702619831Z vllm.engine.async_llm_engine.AsyncEngineDeadError: Background loop is not running. If it was running, inspect the output to find the stacktrace of the error that caused the background loop to stop (AsyncEngineDeadError).
+2026-07-23T03:51:04.702622281Z 
+2026-07-23T03:51:04.702623917Z The above exception was the direct cause of the following exception:
+2026-07-23T03:51:04.702625718Z 
+2026-07-23T03:51:04.702627946Z Traceback (most recent call last):
+2026-07-23T03:51:04.702643211Z   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
+2026-07-23T03:51:04.702645150Z     await app(scope, receive, sender)
+2026-07-23T03:51:04.702646977Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 716, in __call__
+2026-07-23T03:51:04.702648835Z     await self.middleware_stack(scope, receive, send)
+2026-07-23T03:51:04.702650643Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 736, in app
+2026-07-23T03:51:04.702652476Z     await route.handle(scope, receive, send)
+2026-07-23T03:51:04.702654253Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 290, in handle
+2026-07-23T03:51:04.702656095Z     await self.app(scope, receive, send)
+2026-07-23T03:51:04.702657830Z   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 78, in app
+2026-07-23T03:51:04.702659880Z     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
+2026-07-23T03:51:04.702662244Z   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 56, in wrapped_app
+2026-07-23T03:51:04.702665694Z     raise RuntimeError("Caught handled exception, but response already started.") from exc
+2026-07-23T03:51:04.702667893Z RuntimeError: Caught handled exception, but response already started.
+2026-07-23T03:51:04.702669731Z 
+2026-07-23T03:51:04.702671340Z The above exception was the direct cause of the following exception:
+2026-07-23T03:51:04.702673239Z 
+2026-07-23T03:51:04.702674862Z Traceback (most recent call last):
+2026-07-23T03:51:04.702677233Z   File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
+2026-07-23T03:51:04.702679161Z     result = await app(  # type: ignore[func-returns-value]
+2026-07-23T03:51:04.702680952Z   File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
+2026-07-23T03:51:04.702682811Z     return await self.app(scope, receive, send)
+2026-07-23T03:51:04.702684557Z   File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 1082, in __call__
+2026-07-23T03:51:04.702686691Z     await super().__call__(scope, receive, send)
+2026-07-23T03:51:04.702688431Z   File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 113, in __call__
+2026-07-23T03:51:04.702690476Z     await self.middleware_stack(scope, receive, send)
+2026-07-23T03:51:04.702692252Z   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
+2026-07-23T03:51:04.702694354Z     raise exc
+2026-07-23T03:51:04.702696382Z   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
+2026-07-23T03:51:04.702698525Z     await self.app(scope, receive, _send)
+2026-07-23T03:51:04.702700285Z   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in __call__
+2026-07-23T03:51:04.702705835Z     await self.app(scope, receive, send)
+2026-07-23T03:51:04.702708304Z   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
+2026-07-23T03:51:04.702710205Z     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
+2026-07-23T03:51:04.702712202Z   File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 56, in wrapped_app
+2026-07-23T03:51:04.702714127Z     raise RuntimeError("Caught handled exception, but response already started.") from exc
+2026-07-23T03:51:04.702716055Z RuntimeError: Caught handled exception, but response already started.
+2026-07-23T03:51:04.713173377Z CRITICAL 07-23 03:51:04 launcher.py:88] AsyncLLMEngine is already dead, terminating server process
+2026-07-23T03:51:04.714970623Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
+2026-07-23T03:51:04.718637314Z CRITICAL 07-23 03:51:04 launcher.py:88] AsyncLLMEngine is already dead, terminating server process
+2026-07-23T03:51:04.718694821Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
+2026-07-23T03:51:04.722498257Z CRITICAL 07-23 03:51:04 launcher.py:88] AsyncLLMEngine is already dead, terminating server process
+2026-07-23T03:51:04.722585244Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
+2026-07-23T03:51:04.735233268Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:51:04.756590608Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:51:04.768186558Z CRITICAL 07-23 03:51:04 launcher.py:88] AsyncLLMEngine is already dead, terminating server process
+2026-07-23T03:51:04.768268921Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
+2026-07-23T03:51:04.781736020Z INFO:     172.28.22.23:61225 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
+2026-07-23T03:51:04.791948299Z INFO:     Shutting down
+2026-07-23T03:51:04.892200042Z INFO:     Waiting for application shutdown.
+2026-07-23T03:51:04.892346436Z INFO:     Application shutdown complete.
+2026-07-23T03:51:04.893243235Z INFO:     Finished server process [1]
+2026-07-23T03:51:06.253648359Z INFO 07-23 03:51:06 multiproc_worker_utils.py:121] Killing local vLLM worker processes
+2026-07-23T03:51:10.732959761Z Future exception was never retrieved
+2026-07-23T03:51:10.732970009Z future: <Future finished exception=OutOfMemoryError('CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacty of 31.72 GiB of which 50.92 MiB is free. Of the allocated memory 30.86 GiB is allocated by PyTorch, and 210.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF')>
+2026-07-23T03:51:10.732975335Z Traceback (most recent call last):
+2026-07-23T03:51:10.732983365Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 466, in chat_completion_stream_generator
+2026-07-23T03:51:10.732985745Z     async for res in result_generator:
+2026-07-23T03:51:10.732987633Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/utils.py", line 458, in iterate_with_cancellation
+2026-07-23T03:51:10.732989646Z     item = await awaits[0]
+2026-07-23T03:51:10.732991495Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 1046, in generate
+2026-07-23T03:51:10.732993455Z     async for output in await self.add_request(
+2026-07-23T03:51:10.732995398Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 114, in generator
+2026-07-23T03:51:10.732997331Z     raise result
+2026-07-23T03:51:10.732999159Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 56, in _log_task_completion
+2026-07-23T03:51:10.733001089Z     return_value = task.result()
+2026-07-23T03:51:10.733002909Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 857, in run_engine_loop
+2026-07-23T03:51:10.733004845Z     result = task.result()
+2026-07-23T03:51:10.733006732Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 776, in engine_step
+2026-07-23T03:51:10.733008659Z     request_outputs = await self.engine.step_async(virtual_engine)
+2026-07-23T03:51:10.733010626Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/engine/async_llm_engine.py", line 348, in step_async
+2026-07-23T03:51:10.733012560Z     outputs = await self.model_executor.execute_model_async(
+2026-07-23T03:51:10.733014593Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/distributed_gpu_executor.py", line 181, in execute_model_async
+2026-07-23T03:51:10.733016583Z     return await self._driver_execute_model_async(execute_model_req)
+2026-07-23T03:51:10.733018783Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 224, in _driver_execute_model_async
+2026-07-23T03:51:10.733020797Z     return await self.driver_exec_model(execute_model_req)
+2026-07-23T03:51:10.733022570Z   File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
+2026-07-23T03:51:10.733024502Z     result = self.fn(*self.args, **self.kwargs)
+2026-07-23T03:51:10.733026262Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/worker_base.py", line 327, in execute_model
+2026-07-23T03:51:10.733028655Z     output = self.model_runner.execute_model(
+2026-07-23T03:51:10.733030678Z   File "/usr/local/corex/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
+2026-07-23T03:51:10.733032715Z     return func(*args, **kwargs)
+2026-07-23T03:51:10.733034659Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/worker/model_runner.py", line 1679, in execute_model
+2026-07-23T03:51:10.733039337Z     hidden_or_intermediate_states = model_executable(
+2026-07-23T03:51:10.733041153Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:10.733043068Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:10.733044905Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:10.733046777Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:10.733050243Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1421, in forward
+2026-07-23T03:51:10.733053412Z     hidden_states = self.model(
+2026-07-23T03:51:10.733057551Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:10.733059640Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:10.733061352Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:10.733063252Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:10.733065023Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1165, in forward
+2026-07-23T03:51:10.733067165Z     hidden_states, residual = layer(
+2026-07-23T03:51:10.733069092Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:10.733071000Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:10.733072984Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:10.733074867Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:10.733076607Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/models/qwen3_5.py", line 1086, in forward
+2026-07-23T03:51:10.733078748Z     hidden_states, residual = self.post_attention_layernorm(
+2026-07-23T03:51:10.733080781Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
+2026-07-23T03:51:10.733082662Z     return self._call_impl(*args, **kwargs)
+2026-07-23T03:51:10.733084425Z   File "/usr/local/corex/lib/python3/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
+2026-07-23T03:51:10.733086295Z     return forward_call(*args, **kwargs)
+2026-07-23T03:51:10.733088148Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
+2026-07-23T03:51:10.733090040Z     return self._forward_method(*args, **kwargs)
+2026-07-23T03:51:10.733091807Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 187, in forward_cuda
+2026-07-23T03:51:10.733093814Z     return self.forward_native(x, residual)
+2026-07-23T03:51:10.733095539Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 172, in forward_native
+2026-07-23T03:51:10.733101307Z     return self.forward_static(self.weight.data, self.variance_epsilon, x,
+2026-07-23T03:51:10.733104565Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/model_executor/layers/layernorm.py", line 157, in forward_static
+2026-07-23T03:51:10.733107808Z     x = x.float()
+2026-07-23T03:51:10.733109873Z torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacty of 31.72 GiB of which 50.92 MiB is free. Of the allocated memory 30.86 GiB is allocated by PyTorch, and 210.29 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
+2026-07-23T03:51:10.735532351Z Task exception was never retrieved
+2026-07-23T03:51:10.735545848Z future: <Task finished name='Task-129921' coro=<MultiprocessingGPUExecutorAsync._start_worker_execution_loop() done, defined at /usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py:253> exception=OutOfMemoryError('CUDA out of memory. Tried to allocate 32.00 MiB. GPU 2 has a total capacty of 31.72 GiB of which 70.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF')>
+2026-07-23T03:51:10.735550322Z Traceback (most recent call last):
+2026-07-23T03:51:10.735552351Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 258, in _start_worker_execution_loop
+2026-07-23T03:51:10.735554626Z     return await asyncio.gather(*coros)
+2026-07-23T03:51:10.735556397Z   File "/usr/local/corex/lib/python3/dist-packages/vllm/executor/multiproc_worker_utils.py", line 183, in execute_method_async
+2026-07-23T03:51:10.735558462Z     return await future
+2026-07-23T03:51:10.735560338Z torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 2 has a total capacty of 31.72 GiB of which 70.92 MiB is free. Of the allocated memory 30.89 GiB is allocated by PyTorch, and 210.30 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
\ No newline at end of file