feat(SM70): add 1Cat-vLLM FlashQLA fused GDN CUDA kernel for BI-V100

Source: github.com/1CatAI/1Cat-vLLM (MIT license)
flash_qla/ops/gated_delta_rule/chunk/sm70/

Files added:
- csrc/gdn_forward.cu (1919 lines) — 4 CUDA kernels for SM70/SM75:
  gdn_forward, gdn_forward_vlk_varlen,
  gdn_decode_mixed_qkv_global_state, gdn_decode_mixed_qkv_ddtree_state
- fused_fwd.py — Python wrapper, JIT compiles via torch.utils.cpp_extension.load()
- naive_gdn.py — fla reference PyTorch implementation for fallback
- __init__.py — exports chunk_gated_delta_rule_fwd_sm70

Build: JIT compiled at runtime (TORCH_CUDA_ARCH_LIST=7.0;7.5 -O3)
Deploy: patch_ops.sh copies flash_qla_sm70/ to vllm models dir

qwen3_5.py updated to try import flash_qla_sm70 before PyTorch fallback
This commit is contained in:
Claude
2026-08-10 01:06:32 +00:00
parent 3d5f75fefd
commit 8cf73ad39c
6 changed files with 2621 additions and 5 deletions

View File

@@ -76,19 +76,35 @@ _corex_moe_module = None
_corex_gdn_available = False
_corex_moe_available = False
# SM70 FlashQLA GDN kernel (from 1Cat-vLLM, MIT license)
# Fused CUDA kernel for GatedDeltaNet on SM70/SM75 (V100/BI-V100)
# JIT compiled via torch.utils.cpp_extension.load() on first call
_flash_qla_sm70 = None
_flash_qla_available = False
try:
from vllm.model_executor.models.flash_qla_sm70 import (
chunk_gated_delta_rule_fwd_sm70,
chunk_gated_delta_rule_fwd_sm70_vlk_varlen,
)
_flash_qla_available = True
logger.info("FlashQLA SM70 GDN module found — fused CUDA kernel available (JIT on first call)")
except ImportError as e:
logger.warning("FlashQLA SM70 GDN not found (%s) — using PyTorch GDN", e)
try:
from vllm.model_executor.models import corex_gdn as _corex_gdn_module
_corex_gdn_available = True
logger.info("CoreX GDN module found — fused GDN kernels available")
except ImportError:
pass # expected if not packaged; ixformer ops used instead
pass
try:
from vllm.model_executor.models import corex_moe as _corex_moe_module
_corex_moe_available = True
logger.info("CoreX MoE module found — fused MoE kernels available")
except ImportError:
pass # expected; MoE uses PyTorch loop
pass
# ---------------------------------------------------------------------------