Root cause: When tool_choice=auto + tools present, the model enters
<think>...</think> mode by default. On BI-V100 hardware, decode is slow
enough that thinking consumes the entire max_tokens budget, and the model
finishes (finish=stop) before ever emitting <tool_call> XML.
Sub168 reference: d03 in 2.12s with tools=1, finish=tool_calls
Our sub509: d03 in 49.04s with tools=0, finish=stop — FAIL
Fix: Two-layer defense:
1. protocol.py normalize_messages: when tools active + tool_choice=auto
and thinking not explicitly set, auto-set enable_thinking=False
2. qwen3coder_tool_parser.py adjust_request: same logic as defense-in-depth
3. baseline.muh synced with actual computility-run.yaml
Root cause of job 105 scoring 0.0:
- baseline.muh had max_model_len=256000 + gpu_memory_utilization=0.95
- computility-run.yaml had the safe values (100000 + 0.90)
- Platform scheduler sent baseline.muh values to docker run command
- Result: OOM on KV cache allocation → service crash → 881/881 Connection refused
Diagnosis from submit日志:
- benchmark-agent marked success (model loaded OK)
- But service crashed before evaluation started
- All 881 replay requests → Connection refused
- All 5 opencompass benchmarks → 0.0 (aime, gpqa, hle, simpleqa, longbench)
Fix: sync baseline.muh to match computility-run.yaml safe values
FOUND: baseline.muh had completely different values from computility-run.yaml
(the actual deployment config). This means gen_yaml.py would produce a WRONG
computility-run.yaml if someone regenerated it from baseline.muh.
Key differences synced:
max_model_len: 100000 → 256000 (competition allows 256K context)
gpu_memory_utilization: 0.9 → 0.95 (squeeze more KV cache)
max_num_seqs: 1 → 2 (allow 2 concurrent sequences)
max_num_batched_tokens: 8192 → 4096 (smaller prefill chunks)
enforce_eager: (missing) → true (BI-V100 doesn't support CUDA graph)
dtype: (missing) → half
VLLM_ATTENTION_BACKEND: (missing) → XFORMERS
CRITICAL DISCOVERY: CoreX native libraries revealed:
libcorex_fa2.so — Iluvatar FlashAttention2 (NOT generic xformers)
libcorex_gdn.so — CoreX GDN ops
libcorex_moe.so — CoreX MoE GEMM kernel
These are the REAL performance-critical kernels, loaded via VLLM_COREX_*
env vars. The Triton flash_attention.py is a FALLBACK, not the primary path.
CCCL insight: thread_store.cuh shows PTX cache modifiers (st.cg, st.cs)
may be ignored on non-NVIDIA hardware. This explains why LOAD_DEFAULT
outperforms LOAD_LDG on BI-V100 — CoreX has a different cache hierarchy.
Problems fixed:
1. gen_patch.py was reading .muh YAML (all nulls) instead of C++ headers.
Now it parses bi100_* structs directly from tuning_*.cuh via regex,
extracts constexpr values, and maps them to vllm injection points.
Verified: 11 patches generated from 6 algorithms.
2. C++ headers had no build system or tests.
Added CMakeLists.txt (header-only library target) and compile_test.cpp.
Verified: g++ -std=c++17 compiles all headers, 17/17 runtime checks pass.
Also added cuda_compile_test.cu for when nvcc is available.
3. baseline.muh had a tuning section full of nulls duplicating C++ values.
Stripped to vllm launch config only. Tuning values live exclusively
in muh/include/muh/tuning/tuning_*.cuh bi100_* structs.
4. Fixed constexpr goto in tuning_scan.cuh (C++17 doesn't allow goto in
constexpr; replaced with early-return + default: break pattern).
Data flow is now:
tuning_*.cuh (bi100_* constexpr) ──→ gen_patch.py ──→ vllm patches
baseline.muh (launch config) ──→ gen_yaml.py ──→ computility-run.yaml
compile_test.cpp ──→ g++/nvcc ──→ verify values are real
