Commit Graph

5 Commits

Author SHA1 Message Date
project_6
e36da2efa9 [docs+code] lookback delay is a no-op on BI-V100 + V2 compound reduce pattern
Two findings from reading CCCL source code:

1. single_pass_scan_operators.cuh: delay() has GridThreshold=500 gate.
   BI-V100 scan launches ~12 blocks (100K elements / tile_size).
   12 < 500, so ALL delay policies collapse to __threadfence_block().
   Conclusion: delay_ns, delay_l2w, delay_algorithm are IRRELEVANT
   on BI-V100. Only threads/items/load/scan algorithms matter.

2. summary_statistics.cu compound reduce pattern maps directly to
   paged_attention V2's cross-partition reduce. Updated muh_kernel_map.py
   with the structural mapping and the V2 dispatch bug (use_v1=True
   hardcoded in paged_attn.py line 99).

Source: cccl_upstream/cub/cub/agent/single_pass_scan_operators.cuh
        cccl_upstream/thrust/examples/summary_statistics.cu
2026-08-05 03:35:07 +00:00
muh-bot
162a45d4ea [muh_kernel_map] fix syntax error + add bytes_in_flight from CCCL babelstream benchmark
Two missing commas in BI_V100 dict caused Python SyntaxError.

Added bytes_in_flight=64KB from bench_bi100.py real-hardware data:
  bif=8 (64KB) beat bif=0 (32KB) at all problem sizes
  Top 30 transform/float16 results ALL have bif=8
  Physical: BW/SM × HBM_latency = 56 GB/s × 1100ns ≈ 62KB
  CCCL cc_to_min_bytes_in_flight: B200=64KB (54 GB/s/SM)
  BI-V100 matches B200 tier, not A100 (16KB) or V100 (12KB)

Source: cccl_upstream/cub/benchmarks/bench/transform/babelstream.cu
and cccl_upstream/cub/cub/device/dispatch/tuning/tuning_transform.cuh
cc_to_min_bytes_in_flight() function.
2026-08-05 03:34:12 +00:00
root
6beb497447 fix(hardware): SM count 50→16 confirmed on Phanthy Cloud BI-V100
ixsmi + torch.cuda.get_device_properties confirmed:
- multi_processor_count: 16 (not 50 as in spec sheet)
- compute_capability: 7.0 (Volta-compatible)
- max_threads_per_SM: 8192
- total_memory: 32GB per GPU
- SM clock: 1500MHz (max 2500MHz)

Impact: bandwidth_per_sm = 900/16 = 56.25 GB/s (was 18 GB/s at 50 SM)
All occupancy and tile-size calculations need revision.
2026-08-01 13:39:28 +00:00
dylanyunlon
3ebc37d80d [muh] fix scale_mem_bound: 3 bugs vs CCCL util_arch.cuh
1. Return order: (items, threads) not (threads, items) — matches CCCL scaling_result
2. Items clamp upper bound: nominal*2, not nominal*1 — allows small types to double
3. Threads SMEM cap: min(nominal, round_up(max_smem/(type*items), 32)) — prevents SMEM overflow

Verified against all 18 CCCL test cases in catch2_test_util_arch.cu (was 4/14, now 18/18).

Note: C++ tuning headers (tuning_reduce.cuh etc.) have corresponding auto [t, i] destructuring
that also needs to flip to auto [i, t]. The bi100_* struct values themselves are correct
(hand-derived from SMEM constraints), but the policy_selector callers of scale_mem_bound
will produce wrong destructuring. Tracked in project/6 as separate fix item.
2026-08-01 00:00:02 +08:00
Claude
173c6afe09 [muh] kernel_map: full vllm→CCCL mapping with SMEM overflow detection
muh_kernel_map.py maps every vllm kernel to its CCCL algorithm(s):
  paged_attention_v1 → reduce (compound: summary_statistics pattern)
  paged_attention_v2 → reduce + scan (two-pass partitioned)
  sampling_topk → topk + radix_sort
  activation_kernels → transform (SiLU/GELU)
  layernorm_kernels → reduce + transform (variance + normalize)
  rotary_embedding → for_each + transform (RoPE)
  cache_kernels → batch_memcpy (KV block copy)

Found 5 lookahead SMEM overflows — documented in SPECIALIZATION_ANALYSIS.md.
These are non-functional (BI-V100 lacks warpspeed pipeline) but the
dispatch correctly falls back to lookback.

The competitive moat:
  Others: tune 5 vllm launch params → hours
  Us: tune 7 CUB primitive dimensions per algorithm × 6 algorithms,
      constrained by SMEM/occupancy/L2, with CCCL benchmark protocol
2026-07-31 11:13:33 +00:00