fix(critical): fold max_completion_tokens + max_num_seqs=2 + max_model_len=80000 + xllm_latest layer import

Sub 655 root causes (confirmed from log analysis):
1. protocol.py: max_completion_tokens never folded into max_tokens
   → 162/881 replay requests rejected 400 (extra_forbidden)
2. max_num_seqs=1 → t2_n_2 test fails (needs n=2)
3. max_model_len=131072 → OOM crash at 62% replay, opencompass all 0

Fixes:
- protocol.py: model_validator fold_max_completion_tokens
- yaml: max_num_seqs=2, max_model_len=80000, PYTORCH_CUDA_ALLOC_CONF
- topk_softmax stays =0 (corex CUB BlockReduce incompatible on BI-V100)

xllm_latest import to ex_engine/:
- npu_torch layers: GDN(1164L), Qwen3.5 GDN, attention, fused_moe
- cuda/moe kernels: topk_softmax_kernels.cuh, moe_combine, moe_compute_index
- npu kernels: causal_conv1d, recurrent_gated_delta_rule
- model headers: qwen3_5.h, qwen3_next.h
This commit is contained in:
project6-dev
2026-08-13 03:19:39 +00:00
parent a3c45d3b36
commit 0b0c47fddd
44 changed files with 8227 additions and 2 deletions

View File

@@ -8,14 +8,14 @@ command:
- --served-model-name
- llm
- --max-model-len
- '131072'
- '80000'
- --gpu-memory-utilization
- '0.90'
- --trust-remote-code
- -tp
- '4'
- --max-num-seqs
- '1'
- '2'
- --disable-log-requests
- --disable-frontend-multiprocessing
- --max-num-batched-tokens
@@ -47,3 +47,5 @@ env:
value: hybrid64
- name: BI100_MOE_COREX_TOPK_SOFTMAX
value: '0'
- name: PYTORCH_CUDA_ALLOC_CONF
value: expandable_segments:True