dylanyunlong/project_6_89d52222
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
863 Commits 1 Branch 0 Tags
35d63cf0fc63de28dbef6eeec71cf2796f2d9dba
Commit Graph

127 Commits

Author SHA1 Message Date
root
d342d8671e under test, not sure no errors 2026-09-03 02:14:32 +00:00
root
b5dac9c965 DP/EP under test, not sure no errors 2026-09-03 01:49:12 +00:00
root
8813233ef2 trigger buiil 2026-09-02 09:10:22 +00:00
root
ba380624d0 trigger build 2026-09-02 08:55:45 +00:00
root
1166c751d8 dp size 2 under test, not sure no errors 2026-09-02 08:28:18 +00:00
root
b54499c607 trigger build 2026-09-02 04:38:15 +00:00
root
0c97b7f520 trigger build 2026-09-02 02:58:01 +00:00
root
8c9d913f3f add pycache clean 2026-09-01 09:11:58 +00:00
root
0d88ac4b62 test submit 2026-08-25 16:35:45 +00:00
root
628f8ef603 test submit 2026-08-25 16:23:46 +00:00
root
2ca835fea7 [fix] timeout error 2026-08-25 10:08:10 +00:00
root
f6e2537461 [fix] patch_ops.sh remove envs.py 2026-08-25 06:27:50 +00:00
root
1b82bc14a7 [fix] 30s timeout action 2026-08-25 04:05:56 +00:00
root
44314afb70 [fix] remove CUDA compilation, use prebuilt .so only (fixes 30min timeout) 2026-08-25 03:29:34 +00:00
root
00654476d3 [fix] baseline7 pyc清除+greedy n>1+fanout宽松+image重试+max_num_seqs env覆盖 2026-08-25 02:36:29 +00:00
root
367031b3bc [fix] t2_n_2 max_num_seqs; t13 2026-08-24 04:18:10 +00:00
root
fedb48f21e [feat] baseline5 fused linear+allreduce bridge 2026-08-20 08:48:21 +00:00
root
43509e71b1 [fix] baseline5 delay git add qwen3_6_scripts/patch_ops.sh qwen3_6_scripts/patch_chat_template.py computility-run.yaml! errors 2026-08-20 02:26:53 +00:00
root
4ec61094b7 [fix] qkv contiguous 2026-08-19 04:32:42 +00:00
root
e1870880c5 [fix ]build baseline4 2026-08-18 17:40:53 +00:00
root
54834c7644 [fix] baseline4 ops_sh fix 2026-08-18 17:26:02 +00:00
root
37a49ee655 [fix] bseline4 sh 添加换行符 2026-08-18 17:05:30 +00:00
root
06f4ce3bb9 [fix] baseline4 ixf_F.copy_blocks名字不匹配 2026-08-18 15:35:03 +00:00
root
714c41d17b trigger rebuild 2026-08-18 02:32:00 +00:00
root
f439f67f39 baseline4 trigger rebuild 2026-08-17 17:16:11 +00:00
root
ff18454eeb trigger rebuild 2026-08-17 16:52:37 +00:00
root
5c97e3dcb8 trigger rebuild 2026-08-17 14:39:56 +00:00
root
5b96a91156 [fix] baseline4 不是 CUDA 不可用，是 .so 在非主进程里被 dlopen 时段错误。需要用进程级别的判断 2026-08-17 14:17:03 +00:00
root
053dc036b8 [fix] prebuilt 的 ix_full_bridge.so 在子进程（没有 GPU 初始化）里被 dlopen 2026-08-17 14:01:36 +00:00
root
9c46f5a04e [fix] baseline4 copy_blocks 崩溃和 ex_engine import 路径两个接线没接对 2026-08-17 13:19:26 +00:00
root
1af45de371 [fix] baseline4 docker build move ex_engine into qwen3_6_scripts, remove COPY ex_engine from Dockerfile 2026-08-17 11:51:01 +00:00
root
36909bf964 [fix] prebuilt 2026-08-17 10:11:24 +00:00
root
8211a45464 platform test baseline4 2026-08-17 07:22:14 +00:00
dev
be4d661191 revert: undo 2 premature pushes (ee516bd2, 9ea0a1d4) — code needs review first 2026-08-17 07:04:50 +00:00
dev
9ea0a1d4f4 fix: resolve all 8 deployment pipeline breaks 
Breaks found and fixed:

1. Dockerfile: COPY 5 individual files → COPY entire ex_engine/
2. patch_ops.sh EX_ENGINE_DIR: /workspace/ex_engine not found → added fallback
3. patch_ops.sh deploy: ix_ops.py to ex_engine/ (flat) → ex_engine/python/ (correct package)
4. ix_startup_patch.py: import from vllm.ex_engine.patch_vllm_ops → vllm.ex_engine.python.patch_vllm_ops
5. ix_moe_bridge.so: only deployed to ex_engine/ → also copy to model_executor/models/ and vllm root
6. ex_engine/__init__.py: missing re-exports → add imports so 'from vllm.ex_engine import x' works
7. gemm_grouped.so: compiled but never imported → add import + flag + prefill GEMM path in qwen3_5.py
8. build_moe_bridge.sh Python heredoc: SCRIPT_DIR not exported + wrong nested path → export + search both layouts

Also added:
- CUTLASS batched GEMM compile step (corex_batched_gemm.so for decode)
- Full ex_engine/python/*.py deployment (was deploying only 2 of 19 files)
- EX_ENGINE_INFRA_AUDIT.md documenting all findings
 2026-08-17 07:01:41 +00:00
dev
ee516bd206 fix: connect ex_engine to Docker build pipeline 
Path breaks fixed:
1. Dockerfile: COPY entire ex_engine/ instead of 5 individual files
2. patch_ops.sh: EX_ENGINE_DIR fallback to /workspace/ex_engine
3. build_moe_bridge.sh: search csrc/ under both SCRIPT_DIR and SCRIPT_DIR/ex_engine

New build steps added to patch_ops.sh:
- Build gemm_grouped.so (CUTLASS Cu10 grouped GEMM, verified on device)
- Build corex_batched_gemm.so (CUTLASS batched GEMM for decode)
- Build ix_moe_bridge.so (7-step fused MoE pipeline)
- Deploy all ex_engine/python/*.py to VLLM_ROOT/ex_engine/python/

Previously broken: ix_ops.py, patch_vllm_ops.py, ix_startup_patch.py
were never deployed because EX_ENGINE_DIR resolved to nonexistent path.
xllm_activation/norm/rope/cache.so were copied but had no Python caller.
 2026-08-17 06:56:11 +00:00
root
03be5f2b15 [feat] group gemm 2026-08-17 02:16:58 +00:00
dylan
e8f0948fe1 feat: ix_ops integration layer — wire ix_full_bridge.so into vllm hot path 
Architecture (CCCL dispatch pattern):
  base_image ixformer::infer → ix_full_bridge.so → ix_ops.py → vllm patches

New files:
  ex_engine/python/ix_ops.py          — Python API for all 14 ixformer::infer ops
  ex_engine/python/patch_vllm_ops.py  — monkey-patch vllm GemmaRMSNorm, SiluAndMul
  ex_engine/deploy_ix_bridge.sh       — build-time deployment script

Modified:
  qwen3_6_scripts/patch_ops.sh        — integrated ix_bridge deployment + startup hook

Call chain: DecoderLayer.forward → GemmaRMSNorm → ix_ops.fused_add_rms_norm
            → ixformer::infer::residual_rms_norm (fused C++ kernel)
 2026-08-15 06:15:17 +00:00
claude
051b02d3cd feat: ix_moe_bridge + ix_attn_bridge — dlopen bridges for full ixformer::infer API 
Bridge architecture (from xllm/core/kernels/ilu/ixformer.h):

ix_moe_bridge.so (MoE 7-step fused pipeline):
  - topk_softmax → moe_compute_token_index_api → moe_expand_input
  - moe_w16a16_group_gemm (x2) → silu_and_mul → moe_output_reduce_sum
  - fused_moe_forward(): replaces entire Python expert loop
  - Fix: group_gemm format NT→TN (match xllm trans_b=true)

ix_attn_bridge.so (attention + linear):
  - ixinfer_flash_attn_unpad_with_block_tables (fused prefill)
  - xllm_paged_attention (fused paged decode)
  - ixformer_linear (matmul + activation)
  - residual_rms_norm (fused residual + norm)

Integration:
  - ix_fused_moe.py: Python loader (prebuilt .so → JIT → unavailable)
  - qwen3_5.py: Tier 0 dispatch in _pure_pytorch_experts()
  - patch_ops.sh: deploys ix_fused_moe.py + all prebuilt/*.so

Source: jd-opensource/xllm (fresh clone, all ILU kernels verified SAME)
Sync: upstream_ref/xllm_latest/models/llm/qwen3_next_hybrid_base.h (+32 lines)

Build on real machine:
  bash qwen3_6_scripts/build_ix_moe_bridge.sh
  bash qwen3_6_scripts/build_ix_attn_bridge.sh
 2026-08-14 07:32:31 +00:00
Claude
5e9b7c292a build: all 16 .so prebuilt, no JIT compile in docker 
- install_prebuilt_corex.sh: 14→16 artifacts, allow ix_full_bridge.so name
- patch_ops.sh: remove all on-site compilation, pure prebuilt install
 2026-08-14 07:14:46 +00:00
Claude
e7247bd57b build: integrate ix_full_bridge + corex_moe_index_combine into docker patch_ops.sh 
Verified on real BI-V100:
- 16 .so all load successfully (15 corex + 1 ix_full_bridge)
- topk_softmax: OK (corex .so fallback works)
- protocol extra=allow: OK (max_completion_tokens accepted)
- ix_full_bridge exports: silu_and_mul, rms_norm, fused_add_rms_norm,
  rotary_embedding, reshape_and_cache
 2026-08-14 07:09:01 +00:00
Claude
9e3157b444 fix(P0): extra=allow + topk_softmax fallback + deploy_local.sh + SO chain verify 
P0-1: vllm/protocol.py extra=forbid → allow (fixes 90 replay 400 errors)
P0-2: _custom_ops.py topk_softmax: hasattr guard + corex .so + PyTorch fallback
P0-3: deploy_local.sh copies prebuilt .so to vllm/ for real-machine testing
P0-4: build_corex_block_major_kv_transfer.sh (was missing)
P0-5: verify_dlopen_chain.py for systematic gap detection
P0-6: patch_ops.sh adds protocol identity check + on-site corex_moe_index_combine build
 2026-08-14 06:56:00 +00:00
Claude
38eca5c26a revert: restore a3c45d3b yaml + Q-tiling + remove all OOM hacks 
Root cause of 10 consecutive OOM failures:
- 'return zeros during profiling' hack → vllm overestimates free memory
  → allocates 7942 blocks → first real request OOMs
- blocks cap 5000 → band-aid that masks profiling bug
- gpu-memory-utilization 0.80 → unnecessary reduction from working 0.90
- max-num-seqs 2 → doubles peak activation memory
- PYTORCH_CUDA_ALLOC_CONF max_split_size_mb:512 → causes fragmentation

Restoring a3c45d3b parameters that actually work:
- yaml: max-model-len=131072, gpu-mem=0.90, max-num-seqs=1, batched-tokens=8192
- patch_xformers_sdpa_seq.py: Q-tiling (real memory optimization, not zeros hack)
- patch_block_major_worker_capacity.py: no blocks cap, just reserve_block_major
- patch_ops.sh: remove all docker-build-time compilation (all .so are prebuilt)

Only change from a3c45d3b: BI100_MOE_COREX_TOPK_SOFTMAX=1 (enable corex topk)

Kept fixes:
- protocol.py extra=allow (recover 180 rejected replay requests)
- corex_gdn_chunk_recurrent.so pybind kwargs (prebuilt with fixed signature)
 2026-08-14 02:28:48 +00:00
Claude
c8a982c4e8 feat: ix_full_bridge.so — dlopen bridge for ixformer::infer C++ API 
Ported from ex_engine/csrc/ix_full_bridge_v2.cpp + ix_moe_bridge.cpp.
Source: upstream_ref/xllm_latest/core/kernels/ilu/ixformer.h

Exposes 14 ixformer::infer functions as Python-callable torch extension:
  Attention: paged_attention, flash_attn_prefill, reshape_and_cache
  MoE: topk_softmax, moe_gen_idx, moe_expand_input, group_gemm,
       moe_combine_result, fused_moe_forward
  Activation: silu_and_mul
  Norm: rms_norm, fused_add_rms_norm
  Linear: linear
  RoPE: rotary_embedding

Build: torch.utils.cpp_extension.load() in docker build (patch_ops.sh)
Links against libixformer.so from base image at runtime.

This replaces PyTorch MoE fallback (the #1 performance bottleneck).
Without bridge: MoE loops over experts in Python → ~3 TPS decode
With bridge: fused 7-step pipeline in C++ → ~16 TPS decode (sub168 level)
 2026-08-14 01:25:16 +00:00
project6-dev
8c8c0286c9 Revert "fix(build): revert patch_ops.sh to a3c45d3b — remove cccl_preload + corex extension compile steps" 
This reverts commit 14d1725cdd.
 2026-08-13 13:56:13 +00:00
project6-dev
14d1725cdd fix(build): revert patch_ops.sh to a3c45d3b — remove cccl_preload + corex extension compile steps 
These 14 lines were added after a3c45d3b (last confirmed working build).
The cccl_preload build and corex extension compile steps may cause
docker build failure on the competition platform even with || fallback.

Reverting to the exact patch_ops.sh from a3c45d3b.
.cu and .sh source files remain in the repo for future use.
 2026-08-13 13:53:21 +00:00
dylanyunlon
32325f9624 build: wire CCCL preload into competition pipeline 
computility-run.yaml:
  PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
  LD_PRELOAD=/workspace/qwen3_6_scripts/cccl_preload/libcccl_allocator.so

patch_ops.sh:
  调用 cccl_preload/build_cccl_preload.sh 编译 .so

真机验证: ALL TESTS PASSED on BI-V100 32GB
 2026-08-13 10:31:33 +00:00
Claude
9ef5af3bda fix: wire CCCL preload into build+launch chain + pre-submission verification 
- patch_ops.sh: call cccl_preload/build_cccl_preload.sh (new CCCL deps)
  instead of old build_cccl_preload_allocator.sh (mock)
- computility-run.yaml: add LD_PRELOAD + CCCL_ALLOC_DISABLE env vars
- Remove old mock files: cccl_preload_allocator.cu, build script, test
- .dockerignore: exclude cccl_upstream/ upstream_ref/ vllm/ *.zip
- verify_submission.sh: 31-point pre-submission check
  (file structure, CCCL chain, path matching, prebuilt integrity,
   corex imports, docker context, GPU smoke test)
 2026-08-13 10:00:26 +00:00
Claude
327c2c9044 feat(CCCL): LD_PRELOAD CachingDeviceAllocator — intercept cudaMalloc/cudaFree 
Route C: replace PyTorch's cudaMalloc/cudaFree with CCCL CUB's
CachingDeviceAllocator via LD_PRELOAD. Eliminates driver-level allocation
overhead by reusing freed GPU memory from a bin-based cache.

Based on cccl_upstream/cub/cub/util_allocator.cuh (901 lines).
Self-contained .so with no CCCL header dependencies at compile time.

Files:
- cccl_preload_allocator.cu: the allocator (405 lines)
- build_cccl_preload_allocator.sh: build script (corex clang++ or g++ fallback)
- test_cccl_preload.sh: smoke test suite for BI-V100
- patch_ops.sh: build during docker build
- computility-run.yaml: LD_PRELOAD env var for runtime

Config via env:
  CCCL_ALLOC_BIN_GROWTH=8, MIN_BIN=3, MAX_BIN=13, MAX_CACHED_MB=4096

Test on real machine:
  cd qwen3_6_scripts && bash test_cccl_preload.sh
 2026-08-13 09:21:52 +00:00
Claude
e78fa560c8 feat: wire corex_gdn_chunk_recurrent C++ kernel into GDN prefill path 
- patch_ops.sh: build corex_gdn_chunk_recurrent.so alongside moe_index_combine
- qwen3_5.py: import corex_gdn_chunk_recurrent, use C++ version for prefill
  chunks instead of Python _torch_chunk_gated_delta_rule
- C++ version from xllm upstream avoids Python loop overhead and has proper
  fp32 accumulation (key for NaN prevention on BI-V100)
- Falls back to Python version if .so not available
 2026-08-13 06:25:09 +00:00