CUDA: optimize MMQ int8 tensor core performance (#8062)

Browse Source

* CUDA: optimize MMQ int8 tensor core performance

* only a single get_mma_tile_x_k function

* simplify code, make functions constexpr

...

This commit is contained in:

Johannes Gäßler

2024-06-24 12:41:23 +02:00

committed by GitHub

parent 52fc8705a0

commit 9a590c8226

3 changed files with 902 additions and 570 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1412

ggml-cuda/mmq.cuh

File diff suppressed because it is too large Load Diff