project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
Claude	4c796fe4b3	[MUH] Derive BI-V100 tuning values from hardware specs — fix 5 SMEM overflow bugs Previous values were copied verbatim from SM100 (B200). Three of those crash on BI-V100 because tile_size = threads * items * accum_size exceeds the 48KB SMEM limit: REDUCE: float64+o4: SM100(640,16) → tile=81920 > 49152 → BI-V100(512,12) tile=49152 int64+o4: SM100(512,15) → tile=61440 > 49152 → BI-V100(384,16) tile=49152 int64+o8: SM100(512,15) → tile=61440 > 49152 → BI-V100(384,16) tile=49152 SCAN: 8B_o4: SM100(416,23) → tile=76544 > 49152 → BI-V100(416,14) tile=46592 8B_o8: SM100(320,22) → tile=56320 > 49152 → BI-V100(320,19) tile=48640 SCAN DELAY DERIVATION: SM100 L2=50MB, BI-V100 L2=6MB (8.3x smaller cache). Smaller L2 → faster coherence → shorter busy-wait delays. Applied: ns = 0.5, l2w = 0.6 across all 6 lookback tunings. Example: 4B_o4 delay 1904ns→952ns, l2w 830→498. TRANSFORM: min_bytes_in_flight: SM100=64KB but BI-V100 per-SM BW (18 GB/s) matches A100 (18.5 GB/s), not H100/B200. Changed 48KB → 16KB (A100 level). compile_test: 35/35 including SMEM overflow regression test.	2026-07-30 15:08:30 +00:00
Claude	c7a63bc2c8	[MUH] Fix 7 structural discrepancies vs CCCL — read source, not grep Fixes found by reading all 17 muh files + 6 CCCL counterpart policy_selectors as full source code input: 1. topk: BLOCK_LOAD_DIRECT → BLOCK_LOAD_VECTORIZE (CCCL SM90+ uses VECTORIZE). bits_per_pass was wrong (muh: ks<=4→9, CCCL: ks>=2→11). items now computed dynamically (44/key_size) not hardcoded. 2. reduce: added determinism dispatch — three modes matching CCCL: gpu_to_gpu (BLOCK_REDUCE_RAKING, vec_size=1, LOAD_DEFAULT), run_to_run (WARP_REDUCTIONS, LOAD_LDG, default), not_guaranteed (WARP_REDUCTIONS_NONDETERMINISTIC). Added bi100_det_float32 and bi100_det_float64 tuning structs with SM90 benchmark reference values. 3. batch_memcpy: flat single-tier → SmallBuffer+LargeBuffer two-tier matching CCCL structure (128 threads small, 256 threads large, warp_threshold=128, block_threshold=8192). 4. transform: single BulkPolicy → three-policy structure (VectorizedPolicy + AsyncCopyPolicy + PrefetchPolicy) matching CCCL. items_per_thread computed from bytes_in_flight / (threads elem_size). 5. compile_test: 17 checks → 33 checks. Now verifies exact values: reduce determinism modes, topk VECTORIZE + bits=11, batch_memcpy two-tier thresholds, transform three-policy structure. 6. gen_patch: added fallback extraction for inline policy_selector values (topk now generates SAMPLING_BLOCK_SIZE patch). 7. MUH_PROJECT_CHECKPOINT.md: 'PRD设计阶段还没有代码' → actual status. 7 files changed, 413 insertions, 265 deletions.	2026-07-30 14:37:38 +00:00
dylanyunlon	57e222b99d	[MUH] Fix three-layer disconnect — C++ headers are now the single source of truth Problems fixed: 1. gen_patch.py was reading .muh YAML (all nulls) instead of C++ headers. Now it parses bi100_* structs directly from tuning_.cuh via regex, extracts constexpr values, and maps them to vllm injection points. Verified: 11 patches generated from 6 algorithms. 2. C++ headers had no build system or tests. Added CMakeLists.txt (header-only library target) and compile_test.cpp. Verified: g++ -std=c++17 compiles all headers, 17/17 runtime checks pass. Also added cuda_compile_test.cu for when nvcc is available. 3. baseline.muh had a tuning section full of nulls duplicating C++ values. Stripped to vllm launch config only. Tuning values live exclusively in muh/include/muh/tuning/tuning_.cuh bi100_* structs. 4. Fixed constexpr goto in tuning_scan.cuh (C++17 doesn't allow goto in constexpr; replaced with early-return + default: break pattern). Data flow is now: tuning_.cuh (bi100_ constexpr) ──→ gen_patch.py ──→ vllm patches baseline.muh (launch config) ──→ gen_yaml.py ──→ computility-run.yaml compile_test.cpp ──→ g++/nvcc ──→ verify values are real	2026-07-30 14:12:33 +00:00

Author

SHA1

Message

Date

Claude

4c796fe4b3

[MUH] Derive BI-V100 tuning values from hardware specs — fix 5 SMEM overflow bugs

Previous values were copied verbatim from SM100 (B200). Three of those
crash on BI-V100 because tile_size = threads * items * accum_size exceeds
the 48KB SMEM limit:

REDUCE:
  float64+o4: SM100(640,16) → tile=81920 > 49152 → BI-V100(512,12) tile=49152
  int64+o4:   SM100(512,15) → tile=61440 > 49152 → BI-V100(384,16) tile=49152
  int64+o8:   SM100(512,15) → tile=61440 > 49152 → BI-V100(384,16) tile=49152

SCAN:
  8B_o4: SM100(416,23) → tile=76544 > 49152 → BI-V100(416,14) tile=46592
  8B_o8: SM100(320,22) → tile=56320 > 49152 → BI-V100(320,19) tile=48640

SCAN DELAY DERIVATION:
  SM100 L2=50MB, BI-V100 L2=6MB (8.3x smaller cache).
  Smaller L2 → faster coherence → shorter busy-wait delays.
  Applied: ns *= 0.5, l2w *= 0.6 across all 6 lookback tunings.
  Example: 4B_o4 delay 1904ns→952ns, l2w 830→498.

TRANSFORM:
  min_bytes_in_flight: SM100=64KB but BI-V100 per-SM BW (18 GB/s) matches
  A100 (18.5 GB/s), not H100/B200. Changed 48KB → 16KB (A100 level).

compile_test: 35/35 including SMEM overflow regression test.

2026-07-30 15:08:30 +00:00

Claude

c7a63bc2c8

[MUH] Fix 7 structural discrepancies vs CCCL — read source, not grep

Fixes found by reading all 17 muh files + 6 CCCL counterpart
policy_selectors as full source code input:

1. topk: BLOCK_LOAD_DIRECT → BLOCK_LOAD_VECTORIZE (CCCL SM90+ uses
   VECTORIZE). bits_per_pass was wrong (muh: ks<=4→9, CCCL: ks>=2→11).
   items now computed dynamically (4*4/key_size) not hardcoded.

2. reduce: added determinism dispatch — three modes matching CCCL:
   gpu_to_gpu (BLOCK_REDUCE_RAKING, vec_size=1, LOAD_DEFAULT),
   run_to_run (WARP_REDUCTIONS, LOAD_LDG, default),
   not_guaranteed (WARP_REDUCTIONS_NONDETERMINISTIC).
   Added bi100_det_float32 and bi100_det_float64 tuning structs
   with SM90 benchmark reference values.

3. batch_memcpy: flat single-tier → SmallBuffer+LargeBuffer two-tier
   matching CCCL structure (128 threads small, 256 threads large,
   warp_threshold=128, block_threshold=8192).

4. transform: single BulkPolicy → three-policy structure
   (VectorizedPolicy + AsyncCopyPolicy + PrefetchPolicy) matching CCCL.
   items_per_thread computed from bytes_in_flight / (threads * elem_size).

5. compile_test: 17 checks → 33 checks. Now verifies exact values:
   reduce determinism modes, topk VECTORIZE + bits=11, batch_memcpy
   two-tier thresholds, transform three-policy structure.

6. gen_patch: added fallback extraction for inline policy_selector
   values (topk now generates SAMPLING_BLOCK_SIZE patch).

7. MUH_PROJECT_CHECKPOINT.md: 'PRD设计阶段还没有代码' → actual status.

7 files changed, 413 insertions, 265 deletions.

2026-07-30 14:37:38 +00:00

dylanyunlon

57e222b99d

[MUH] Fix three-layer disconnect — C++ headers are now the single source of truth

Problems fixed:
  1. gen_patch.py was reading .muh YAML (all nulls) instead of C++ headers.
     Now it parses bi100_* structs directly from tuning_*.cuh via regex,
     extracts constexpr values, and maps them to vllm injection points.
     Verified: 11 patches generated from 6 algorithms.

  2. C++ headers had no build system or tests.
     Added CMakeLists.txt (header-only library target) and compile_test.cpp.
     Verified: g++ -std=c++17 compiles all headers, 17/17 runtime checks pass.
     Also added cuda_compile_test.cu for when nvcc is available.

  3. baseline.muh had a tuning section full of nulls duplicating C++ values.
     Stripped to vllm launch config only. Tuning values live exclusively
     in muh/include/muh/tuning/tuning_*.cuh bi100_* structs.

  4. Fixed constexpr goto in tuning_scan.cuh (C++17 doesn't allow goto in
     constexpr; replaced with early-return + default: break pattern).

Data flow is now:
  tuning_*.cuh (bi100_* constexpr) ──→ gen_patch.py ──→ vllm patches
  baseline.muh (launch config)     ──→ gen_yaml.py  ──→ computility-run.yaml
  compile_test.cpp                 ──→ g++/nvcc     ──→ verify values are real

2026-07-30 14:12:33 +00:00

3 Commits