Claude
|
e87470733d
|
accel(ixformer): wire BI-V100 hardware primitives into GDN + MoE compute paths
Before: 9 ixformer ops available, 0 used by our code (100% pure PyTorch).
After: matmul/bmm/softmax wired into every hot path.
Decode path (runs for EVERY generated token):
- 2× torch.bmm → _ix_bmm (kv_mem lookup + output projection)
Chunk scan loop (prefill, runs per 2048-token chunk):
- k_beta @ key.T → _ix_matmul
- attn @ v_beta → _ix_matmul
- attn @ k_beta_exp → _ix_matmul
- 6× matmul inside state update loop → _ix_matmul
MoE routing + expert dispatch:
- torch.softmax → _ix_softmax (router)
- torch.bmm in decode fast-path → _ix_bmm
Also adds CODEPATH_MAP.md — complete source-file-level timing diagram
from HTTP request to GPU kernel, with line numbers.
ixformer.matmul signature: matmul(input, other, out, transa, transb, alpha, beta)
ixformer.softmax signature: softmax(input, dim)
Both fall back to torch if ixformer unavailable.
|
2026-08-08 22:35:21 +00:00 |
|