project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
Claude	a7537ebee0	doc: add functional test FAIL root cause analysis to CODEPATH_MAP 6 non-crash FAILs traced to root cause: - 5 are NaN-induced model quality issues (will self-heal with clamp fix) - 1 is multimodal HTTP 400 (needs separate debug) - 25 are crash cascade (will self-heal with max-num-seqs=2) Expected after deployment: 45/51 PASS (88%)	2026-08-09 00:03:45 +00:00
Claude	e87470733d	accel(ixformer): wire BI-V100 hardware primitives into GDN + MoE compute paths Before: 9 ixformer ops available, 0 used by our code (100% pure PyTorch). After: matmul/bmm/softmax wired into every hot path. Decode path (runs for EVERY generated token): - 2× torch.bmm → _ix_bmm (kv_mem lookup + output projection) Chunk scan loop (prefill, runs per 2048-token chunk): - k_beta @ key.T → _ix_matmul - attn @ v_beta → _ix_matmul - attn @ k_beta_exp → _ix_matmul - 6× matmul inside state update loop → _ix_matmul MoE routing + expert dispatch: - torch.softmax → _ix_softmax (router) - torch.bmm in decode fast-path → _ix_bmm Also adds CODEPATH_MAP.md — complete source-file-level timing diagram from HTTP request to GPU kernel, with line numbers. ixformer.matmul signature: matmul(input, other, out, transa, transb, alpha, beta) ixformer.softmax signature: softmax(input, dim) Both fall back to torch if ixformer unavailable.	2026-08-08 22:35:21 +00:00

Author

SHA1

Message

Date

Claude

a7537ebee0

doc: add functional test FAIL root cause analysis to CODEPATH_MAP

6 non-crash FAILs traced to root cause:
- 5 are NaN-induced model quality issues (will self-heal with clamp fix)
- 1 is multimodal HTTP 400 (needs separate debug)
- 25 are crash cascade (will self-heal with max-num-seqs=2)

Expected after deployment: 45/51 PASS (88%)

2026-08-09 00:03:45 +00:00

Claude

e87470733d

accel(ixformer): wire BI-V100 hardware primitives into GDN + MoE compute paths

Before: 9 ixformer ops available, 0 used by our code (100% pure PyTorch).
After: matmul/bmm/softmax wired into every hot path.

Decode path (runs for EVERY generated token):
  - 2× torch.bmm → _ix_bmm (kv_mem lookup + output projection)

Chunk scan loop (prefill, runs per 2048-token chunk):
  - k_beta @ key.T → _ix_matmul
  - attn @ v_beta → _ix_matmul
  - attn @ k_beta_exp → _ix_matmul
  - 6× matmul inside state update loop → _ix_matmul

MoE routing + expert dispatch:
  - torch.softmax → _ix_softmax (router)
  - torch.bmm in decode fast-path → _ix_bmm

Also adds CODEPATH_MAP.md — complete source-file-level timing diagram
from HTTP request to GPU kernel, with line numbers.

ixformer.matmul signature: matmul(input, other, out, transa, transb, alpha, beta)
ixformer.softmax signature: softmax(input, dim)
Both fall back to torch if ixformer unavailable.

2026-08-08 22:35:21 +00:00

2 Commits