project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
dylanyunlon	52c5ca7ce5	refactor(muh_dispatch): read-once from C++ headers, not write-twice Replaces hand-written reduce_threads=512, reduce_items=16 with _read_reduce_config(accum_size) that reads from tuning_reduce.cuh via gen_patch.extract_bi100_structs(). Architecture change: OLD: hand-write values in Python + verify_against_headers() asserts equal NEW: _read_reduce_config() reads from C++ header (single source of truth) Falls back to compiled-in defaults only when headers not on disk (deployed container), with RuntimeWarning. No hand-written tuning values remain in the normal code path. verify_against_headers() removed — there is nothing to verify when there is only one copy of the truth.	2026-08-01 01:30:42 +08:00
dylanyunlon	03f6a59ebf	fix(muh_dispatch): add verify_against_headers() to close the loop Adds verification that hand-written values in muh_dispatch.py (reduce_threads=512, reduce_items=16, etc.) match the C++ headers (bi100_float32_plus_o4 in tuning_reduce.cuh). Previously: muh_dispatch.py had hand-coded values with no link to the C++ source of truth. gen_patch.py reads from C++ headers, but muh_dispatch.py was a separate copy that could diverge. Now: verify_against_headers() calls gen_patch.extract_bi100_structs() and compares. Self-test prints mismatches if any exist.	2026-08-01 00:32:16 +08:00
dylanyunlon	e69c46d0b7	[muh] add muh_dispatch.py — CCCL-style type-dispatched kernel config for BI-V100 This is the key differentiator vs parameter brute-force. Everyone else hardcodes BLOCK_SIZE=64, NUM_WARPS=4, PARTITION_SIZE=512. muh_dispatch replaces these with type-dispatched values derived from CCCL's policy_selector architecture. Dispatch axes (matching CCCL type_t × op_kind_t × offset_size): - dtype → determines accum_size, SMEM per element - head_dim → determines tile width, SMEM constraint - max_seq_len → determines V1/V2 threshold (single_tile vs multi_tile) - num_kv_heads → determines GQA ratio (memory access pattern) Output: AttentionConfig struct with all kernel parameters. CCCL reference: ReducePolicy{multi_tile, single_tile} pattern. Example type dispatches for Qwen3.6 on BI-V100: bf16 h128 100K → partition=512, v1_thresh=8192, reduce(512,16,vec=4) bf16 h256 100K → partition=256, v1_thresh=8192, reduce(512,16,vec=4) fp32 h128 32K → partition=256, v1_thresh=8192, reduce(512,16,vec=4) bf16 h128 2K → v1_thresh=2049 (always V1, skip V2 overhead)	2026-07-31 19:11:46 +08:00

Author

SHA1

Message

Date

dylanyunlon

52c5ca7ce5

refactor(muh_dispatch): read-once from C++ headers, not write-twice

Replaces hand-written reduce_threads=512, reduce_items=16 with
_read_reduce_config(accum_size) that reads from tuning_reduce.cuh
via gen_patch.extract_bi100_structs().

Architecture change:
  OLD: hand-write values in Python + verify_against_headers() asserts equal
  NEW: _read_reduce_config() reads from C++ header (single source of truth)
       Falls back to compiled-in defaults only when headers not on disk
       (deployed container), with RuntimeWarning.

No hand-written tuning values remain in the normal code path.
verify_against_headers() removed — there is nothing to verify
when there is only one copy of the truth.

2026-08-01 01:30:42 +08:00

dylanyunlon

03f6a59ebf

fix(muh_dispatch): add verify_against_headers() to close the loop

Adds verification that hand-written values in muh_dispatch.py
(reduce_threads=512, reduce_items=16, etc.) match the C++ headers
(bi100_float32_plus_o4 in tuning_reduce.cuh).

Previously: muh_dispatch.py had hand-coded values with no link to
the C++ source of truth. gen_patch.py reads from C++ headers,
but muh_dispatch.py was a separate copy that could diverge.

Now: verify_against_headers() calls gen_patch.extract_bi100_structs()
and compares. Self-test prints mismatches if any exist.

2026-08-01 00:32:16 +08:00

dylanyunlon

e69c46d0b7

[muh] add muh_dispatch.py — CCCL-style type-dispatched kernel config for BI-V100

This is the key differentiator vs parameter brute-force.

Everyone else hardcodes BLOCK_SIZE=64, NUM_WARPS=4, PARTITION_SIZE=512.
muh_dispatch replaces these with type-dispatched values derived from
CCCL's policy_selector architecture.

Dispatch axes (matching CCCL type_t × op_kind_t × offset_size):
  - dtype → determines accum_size, SMEM per element
  - head_dim → determines tile width, SMEM constraint
  - max_seq_len → determines V1/V2 threshold (single_tile vs multi_tile)
  - num_kv_heads → determines GQA ratio (memory access pattern)

Output: AttentionConfig struct with all kernel parameters.
CCCL reference: ReducePolicy{multi_tile, single_tile} pattern.

Example type dispatches for Qwen3.6 on BI-V100:
  bf16 h128 100K → partition=512, v1_thresh=8192, reduce(512,16,vec=4)
  bf16 h256 100K → partition=256, v1_thresh=8192, reduce(512,16,vec=4)
  fp32 h128 32K  → partition=256, v1_thresh=8192, reduce(512,16,vec=4)
  bf16 h128 2K   → v1_thresh=2049 (always V1, skip V2 overhead)

2026-07-31 19:11:46 +08:00

3 Commits