EngineX-Ascend/enginex-ascend-910-llama.cpp
10
0
Fork 0
Code Issues Pull Requests Actions 4 Projects Releases Wiki Activity

metal : extend mat-mat multiplication support (#16225)

Browse Source 
* metal : support mul_mm with src1->type == GGML_TYPE_F16

* metal : support mul_mm_id with src1->type == GGML_TYPE_F16

[no ci]

* metal : mul_mm support ne00 % 32 != 0

* metal : support mul_mm_id with ne00 % 32 != 0

* cont : remove unnecessary unrolls

* cont : simplify data loading

* metal : optimize mul_mm when output bounds checks are not needed
This commit is contained in:
Georgi Gerganov
2025-09-28 09:34:44 +03:00
committed by GitHub
parent 3b53634fe3
commit 6a2c6145a0
7 changed files with 247 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
ggml/src/ggml-metal/ggml-metal-impl.h
View File

@@ -76,6 +76,7 @@
#define FC_FLASH_ATTN_EXT_VEC 200
#define FC_FLASH_ATTN_EXT_VEC_REDUCE 300
#define FC_MUL_MV 400
#define FC_MUL_MM 500
// kernel argument structs
//