document multi-step scheduler diagnostics

worklogs/2026-07-13-initial-run.md

@@ -268,3 +268,22 @@
   - Increasing concurrency does not improve aggregate decode throughput and worsens per-request TPS.
   - Low average GPU utilization suggests scheduling, TP synchronization, MoE kernel, paged attention, or xFormers backend limitations before raw compute saturation.
   - Next focused experiment: disable chunked prefill and retry `--num-scheduler-steps 4` as a decode-only diagnostic variant.
+
+## 2026-07-14 Multi-Step Scheduling Diagnostic
+
+- Goal: verify whether `--num-scheduler-steps 4` can improve decode throughput.
+- Appended details to:
+  - `worklogs/decode_analysis_report_2026-07-14.md`
+- Synced remote failure logs to:
+  - `worklogs/remote_results/2026-07-14-scheduler/`
+- Experiments:
+  - No chunked prefill + `max_model_len=100000` failed because `max_num_batched_tokens=8192` is smaller than `max_model_len=100000`.
+  - No chunked prefill + `max_model_len=4096` still failed because `Multi-Step not supported for attention backend: xformers`.
+  - Forced `VLLM_ATTENTION_BACKEND=FLASHINFER` failed because `BatchDecodeWithPagedKVCacheWrapper` was `None`, indicating FlashInfer dependencies/wrappers are unavailable or incompatible.
+- Conclusion:
+  - Multi-step scheduling cannot currently be tested or used on this server because the available attention backend is xFormers.
+  - The next practical direction is not more `num_scheduler_steps` tuning, but either making FlashAttention/FlashInfer backend available or profiling the current xFormers + MoE + TP decode path.
+- Recovery:
+  - Restored full parser service with chunked prefill and prefix cache.
+  - Recovery log: `/root/work/logs/server_exp_seq2_b8192_fullparser_restored_after_schedtest.log`.
+  - `/health` returned `200`.