project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
Claude	f87689a4ef	fix(CRITICAL): engine death on image request + stop overwriting base corex modules Root cause from latest docker build log: ValueError: You set image=0 in --limit-mm-per-prompt, but found 1 items → Engine background task crashes → AsyncEngineDeadError → all subsequent 503 Fixes: 1. computility-run.yaml: add --limit-mm-per-prompt image=1 Prevents multimodal ValueError from killing the engine process. 2. patch_ops.sh: DON'T overwrite base image's corex_gdn.py/corex_moe.py Comp 168 log proves base image's corex modules work with libcorex_gdn.so. Our overwrite broke CoreXGDN.__init__ (unexpected kwarg 'num_v_heads'). Only deploy ours if base has NO corex modules at all. Also deploy corex_fa2.py if base lacks it. 3. qwen3_5.py: try multiple CoreXGDN init signatures Base image CoreXGDN may accept different kwargs than ours. Try kwargs form first, fall back to positional. 4. corex_gdn.py: accept both calling conventions in __init__ Future-proof for when we DO need to deploy ours. 5. Copied upstream_ref headers: ilu_layer_fused_moe.h, ilu_layer_attention.h Last 2 missing ILU files from xllm. All 14/14 now present.	2026-08-10 09:12:05 +00:00
project6-dev	f4e2264a83	ref(EX): import upstream ILU kernels + xllm MoE CUDA sources into ex_engine Copied from upstream_ref (NOT rewritten — exact upstream code): ixformer C++ API (the authoritative header): include/ixformer.h — ixformer::infer namespace: topk_softmax, moe_compute_token_index_api, moe_w16a16_group_gemm, moe_expand_input, moe_output_reduce_sum, silu_and_mul, rms_norm, xllm_paged_attention, etc. include/ilu_ops_api.h — xllm::kernel::ilu namespace: moe_active_topk, moe_gen_idx, moe_expand_input, group_gemm, moe_combine_result, batch_prefill, batch_decode, rms_norm, matmul, act_and_mul, etc. ILU kernel wrappers (call ixformer::infer directly): csrc/ilu_kernel_fused_moe.cpp — topk routing + gen_idx + expand + combine csrc/ilu_kernel_group_gemm.cpp — batched expert GEMM csrc/ilu_kernel_{activation,norm,rope,matmul,attention}.cpp ILU layer implementations (full pipeline): csrc/ilu_layer_fused_moe.{cpp,h} — 797 lines, the complete MoE pipeline that competitor 168 ran as corex_moe.py csrc/ilu_layer_attention.{cpp,h} — prefill/decode attention dispatch CUDA MoE kernels (from xllm + ds_vllm): csrc/moe/moe_topk_softmax_kernels.cuh — CUB BlockReduce + warp topk csrc/moe/moe_topk_sigmoid_kernels.cuh — sigmoid scoring variant csrc/moe/moe_topk.cuh + moe_fused_topk.cu — entry points csrc/moe/moeTopKFuncs.cuh — TRT-LLM derived vllm-compatible topk csrc/moe/moe_ops.h + moe_align_sum_kernels.cu — alignment kernels Common layer headers: csrc/common_fused_moe{,_base}.h + common_moe_fused_topk.{cpp,h}	2026-08-10 03:59:45 +00:00
EX Engine	fcfb764560	feat(EX): Algorithm Factor Replacement Engine — dlopen-based CUDA kernel dispatch Factors: 0 (moe_topk_softmax), 2 (moe_fused_gemm), 5 (gdn_chunk_fwd) Fixes: topk_softmax fallback (2304x/token), GDN NaN (frac=0.9998-1.0)	2026-08-10 02:25:23 +00:00

Author

SHA1

Message

Date

Claude

f87689a4ef

fix(CRITICAL): engine death on image request + stop overwriting base corex modules

Root cause from latest docker build log:
  ValueError: You set image=0 in --limit-mm-per-prompt, but found 1 items
  → Engine background task crashes → AsyncEngineDeadError → all subsequent 503

Fixes:
1. computility-run.yaml: add --limit-mm-per-prompt image=1
   Prevents multimodal ValueError from killing the engine process.

2. patch_ops.sh: DON'T overwrite base image's corex_gdn.py/corex_moe.py
   Comp 168 log proves base image's corex modules work with libcorex_gdn.so.
   Our overwrite broke CoreXGDN.__init__ (unexpected kwarg 'num_v_heads').
   Only deploy ours if base has NO corex modules at all.
   Also deploy corex_fa2.py if base lacks it.

3. qwen3_5.py: try multiple CoreXGDN init signatures
   Base image CoreXGDN may accept different kwargs than ours.
   Try kwargs form first, fall back to positional.

4. corex_gdn.py: accept both calling conventions in __init__
   Future-proof for when we DO need to deploy ours.

5. Copied upstream_ref headers: ilu_layer_fused_moe.h, ilu_layer_attention.h
   Last 2 missing ILU files from xllm. All 14/14 now present.

2026-08-10 09:12:05 +00:00

project6-dev

f4e2264a83

ref(EX): import upstream ILU kernels + xllm MoE CUDA sources into ex_engine

Copied from upstream_ref (NOT rewritten — exact upstream code):

ixformer C++ API (the authoritative header):
  include/ixformer.h — ixformer::infer namespace: topk_softmax,
    moe_compute_token_index_api, moe_w16a16_group_gemm, moe_expand_input,
    moe_output_reduce_sum, silu_and_mul, rms_norm, xllm_paged_attention, etc.
  include/ilu_ops_api.h — xllm::kernel::ilu namespace: moe_active_topk,
    moe_gen_idx, moe_expand_input, group_gemm, moe_combine_result,
    batch_prefill, batch_decode, rms_norm, matmul, act_and_mul, etc.

ILU kernel wrappers (call ixformer::infer directly):
  csrc/ilu_kernel_fused_moe.cpp — topk routing + gen_idx + expand + combine
  csrc/ilu_kernel_group_gemm.cpp — batched expert GEMM
  csrc/ilu_kernel_{activation,norm,rope,matmul,attention}.cpp

ILU layer implementations (full pipeline):
  csrc/ilu_layer_fused_moe.{cpp,h} — 797 lines, the complete MoE pipeline
    that competitor 168 ran as corex_moe.py
  csrc/ilu_layer_attention.{cpp,h} — prefill/decode attention dispatch

CUDA MoE kernels (from xllm + ds_vllm):
  csrc/moe/moe_topk_softmax_kernels.cuh — CUB BlockReduce + warp topk
  csrc/moe/moe_topk_sigmoid_kernels.cuh — sigmoid scoring variant
  csrc/moe/moe_topk.cuh + moe_fused_topk.cu — entry points
  csrc/moe/moeTopKFuncs.cuh — TRT-LLM derived vllm-compatible topk
  csrc/moe/moe_ops.h + moe_align_sum_kernels.cu — alignment kernels

Common layer headers:
  csrc/common_fused_moe{,_base}.h + common_moe_fused_topk.{cpp,h}

2026-08-10 03:59:45 +00:00

EX Engine

fcfb764560

feat(EX): Algorithm Factor Replacement Engine — dlopen-based CUDA kernel dispatch

Factors: 0 (moe_topk_softmax), 2 (moe_fused_gemm), 5 (gdn_chunk_fwd)
Fixes: topk_softmax fallback (2304x/token), GDN NaN (frac=0.9998-1.0)

2026-08-10 02:25:23 +00:00

3 Commits