project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
project6	8b6f3fd242	fix(MoE): robust CUDA kernel loading + no-GPU precompile 1. precompile_moe_topk.py: skip GPU verification during Docker build (torch.cuda.is_available() check — .so compilation doesn't need GPU) 2. _custom_ops.py topk_softmax init: 3-tier loading - import precompiled module (torch cache) - scan known .so paths (torch_extensions cache dirs) - JIT compile from .cu source - PyTorch fallback with WARNING (not silent — must know if CUDA failed) 3. patch_ops.sh: report .so location after precompile for debugging	2026-08-10 07:50:28 +00:00
project6-dev	f32ef97013	feat(MoE): verified CUDA topk_softmax kernel — zero fallback moe_topk_softmax_v3.cu: BI-V100 verified (2026-08-10) - 64 experts, topk=8, warp shuffle, zero shared memory - renormalize: sum=1.0 ✓, no NaN ✓, no duplicate ids ✓ - 881 token batch ✓ - Compiler: corex clang/16, --cuda-gpu-arch=ivcore10 - Stream: c10::cuda::getCurrentCUDAStream() corex_moe.py: loads CUDA kernel, NO Python fallback - Searches pre-compiled .so → JIT compile from source → error - MoE pipeline: CUDA topk → cublas expert GEMM → ixformer silu_and_mul precompile_moe_topk.py: Docker build-time compilation + verification Key finding from real machine probing: ixformer::infer::topk_softmax is DECLARED in ixformer.h but NOT IMPLEMENTED in any .so in the base image (nm -D scan: zero hits). Must compile our own kernel.	2026-08-10 04:21:43 +00:00

Author

SHA1

Message

Date

project6

8b6f3fd242

fix(MoE): robust CUDA kernel loading + no-GPU precompile

1. precompile_moe_topk.py: skip GPU verification during Docker build
   (torch.cuda.is_available() check — .so compilation doesn't need GPU)

2. _custom_ops.py topk_softmax init: 3-tier loading
   - import precompiled module (torch cache)
   - scan known .so paths (torch_extensions cache dirs)
   - JIT compile from .cu source
   - PyTorch fallback with WARNING (not silent — must know if CUDA failed)

3. patch_ops.sh: report .so location after precompile for debugging

2026-08-10 07:50:28 +00:00

project6-dev

f32ef97013

feat(MoE): verified CUDA topk_softmax kernel — zero fallback

moe_topk_softmax_v3.cu: BI-V100 verified (2026-08-10)
  - 64 experts, topk=8, warp shuffle, zero shared memory
  - renormalize: sum=1.0 ✓, no NaN ✓, no duplicate ids ✓
  - 881 token batch ✓
  - Compiler: corex clang/16, --cuda-gpu-arch=ivcore10
  - Stream: c10::cuda::getCurrentCUDAStream()

corex_moe.py: loads CUDA kernel, NO Python fallback
  - Searches pre-compiled .so → JIT compile from source → error
  - MoE pipeline: CUDA topk → cublas expert GEMM → ixformer silu_and_mul

precompile_moe_topk.py: Docker build-time compilation + verification

Key finding from real machine probing:
  ixformer::infer::topk_softmax is DECLARED in ixformer.h but
  NOT IMPLEMENTED in any .so in the base image (nm -D scan: zero hits).
  Must compile our own kernel.

2026-08-10 04:21:43 +00:00

2 Commits