dylanyunlong/project_6

Go to file

dylanyunlon dc9ac0a757 feat(muh): apply CCCL-derived BI-V100 tuning to 5 vllm Python files

Applied via muh/vllm_bi100_patch.py --conservative:

1. paged_attn.py: removed use_v1=True hardcode, restored V1/V2 heuristic
   with BI-V100 threshold (16384 vs default 8192). SM=16 favors V1 longer.

2. fused_moe.py: BLOCK_SIZE_K 32→64 (better memory coalescing with 900GB/s
   BW), BLOCK_SIZE_N 32→64 for decode path. Qwen3.6 MoE: E≈128, topk=8.

3. _custom_ops.py: SMEM kept at 32KB (conservative mode, pending hardware
   confirmation). Added diagnostic comment.

4. prefix_prefill.py: enhanced BI-V100 block config comment with SMEM
   budget breakdown (BLOCK=64,N=64 → 48KB tight, N=32 → 32KB safe).

5. triton_flash_attention.py: added 2 BI-V100 autotune configs
   (64x32 and 32x64) for SM=16 occupancy characteristics.

CCCL basis: cub/benchmarks/bench/ %RANGE% parameter spaces (reduce 1044
combos, scan 5.4M, topk 1698, transform 25920) → SMEM pruning → policy
selector logic from tuning_*.cuh.

Also includes muh/vllm_bi100_patch.py (713 lines) for reproducible
one-shot patching with --dry-run, --conservative, and --revert modes.

2026-08-03 10:27:10 +00:00

[INFRA] Import NVIDIA/CCCL upstream as optimization reference library

2026-07-30 09:35:51 +00:00

docs: CCCL → Triton methodology transfer — parameter search translation table

2026-08-03 04:37:23 +00:00

feat(muh): apply CCCL-derived BI-V100 tuning to 5 vllm Python files

2026-08-03 10:27:10 +00:00

fix: critical config + tuning corrections from CCCL source analysis

2026-08-03 06:45:54 +00:00

qwen3_6_scripts

feat: CCCL-derived 3-tier decode dispatch + SM=16 prefill tuning + multi-step scheduling

2026-08-03 08:28:38 +00:00

feat(muh): apply CCCL-derived BI-V100 tuning to 5 vllm Python files

2026-08-03 10:27:10 +00:00

feat: add qwen3_5.py vllm adapter for Qwen3.6-35B-A3B

2026-08-01 13:54:33 +00:00

__init__.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

.gitignore

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

attention.py

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

baseline.muh

[MUH] Fix three-layer disconnect — C++ headers are now the single source of truth

2026-07-30 14:12:33 +00:00

CCCL_INTEGRATION_STATUS.md

docs: CCCL integration status — 462 assets inventory, 8/8 decode hot path coverage, no-clone-more verdict

2026-08-03 03:50:24 +00:00

chat_dataset_v0.json

[DEPLOY] Complete submission: baseline + all optimizations

2026-07-30 16:06:20 +00:00

computility-run.yaml

feat: CCCL-derived 3-tier decode dispatch + SM=16 prefill tuning + multi-step scheduling

2026-08-03 08:28:38 +00:00

Dockerfile

feat: deploy CCCL-tuned prefix_prefill + muh_dispatch + fix SM=16 count

2026-08-03 08:30:16 +00:00

ENGINEX_INJECTION_MAP.md

docs: EngineX vllm injection map — Python not C++, Triton not CUDA, 32KB SMEM claim

2026-08-03 03:59:49 +00:00

enginex-vllm-bi100-qwen36-main.zip

update baseline

2026-07-30 15:19:37 +00:00

launch_service

sync: align with enginex-vllm-bi100-qwen36 baseline (1902c81f)

2026-07-31 09:43:58 +00:00

muh_dispatch.py

feat: deploy CCCL-tuned prefix_prefill + muh_dispatch + fix SM=16 count

2026-08-03 08:30:16 +00:00

muh_kernel_map.py

fix(hardware): SM count 50→16 confirmed on Phanthy Cloud BI-V100

2026-08-01 13:39:28 +00:00

MUH_PROJECT_CHECKPOINT.md

[MUH] Fix 7 structural discrepancies vs CCCL — read source, not grep

2026-07-30 14:37:38 +00:00

paged_attention_v2_pytorch.py

[FIX] V2 shape mismatch bug — v_padded used num_heads for kv_h tensor

2026-07-31 03:52:23 +00:00

paged_attention_v2_triton.py

[ARCH] CCCL-derived paged attention kernel architecture + Triton rewrite

2026-07-31 04:13:07 +00:00

paged_attn.py

sync: align with enginex-vllm-bi100-qwen36 baseline (1902c81f)

2026-07-31 09:43:58 +00:00

prefix_prefill.py

feat(muh): apply CCCL-derived BI-V100 tuning to 5 vllm Python files

2026-08-03 10:27:10 +00:00

README.md

Initial commit

2026-07-30 17:03:23 +08:00

README.md

project_6

Languages

C++ 41.8%

Cuda 31.6%

Python 22.2%

C 2.1%

CMake 1.1%

Other 1.1%