284804ac53
Sources: siboehm/SGEMM_CUDA → upstream_ref/sgemm_siboehm/ (25 files) wangzyon/NVIDIA_SGEMM_PRACTICE → upstream_ref/nvidia_sgemm_practice/ (23 files, filled gaps) edtallison/sgemm-cuda → upstream_ref/sgemm_edtallison/ (41 files) All files cat'd one by one from git clone (no --depth). These are the 3 public SGEMM repos that can compile on CUDA 10.2 + CoreX ivcore10. Key files for BI-V100 porting: kernel 10 (warp tiling) — already proven on device with WARPSIZE=64 kernel 11/12 (double buffering) — next optimization target sgemm.cu + runner.cu — complete build+benchmark harness CMakeLists.txt — build system reference
43 lines
2.3 KiB
Markdown
43 lines
2.3 KiB
Markdown
# Fast CUDA SGEMM from Scratch
|
|
|
|
Step-by-step optimization of matrix multiplication, implemented in CUDA.
|
|
For an explanation of each kernel, see [siboehm.com/CUDA-MMM](https://siboehm.com/articles/22/CUDA-MMM).
|
|
|
|
## Overview
|
|
|
|
Running the kernels on a NVIDIA A6000 (Ampere):
|
|
|
|
|
|
|
|
GFLOPs at matrix size 4096x4096:
|
|
<!-- benchmark_results -->
|
|
| Kernel | GFLOPs/s | Performance relative to cuBLAS |
|
|
|:------------------------------------|----------:|:-------------------------------|
|
|
| 1: Naive | `309.0` | 1.3% |
|
|
| 2: GMEM Coalescing | `1986.5` | 8.5% |
|
|
| 3: SMEM Caching | `2980.3` | 12.8% |
|
|
| 4: 1D Blocktiling | `8474.7` | 36.5% |
|
|
| 5: 2D Blocktiling | `15971.7` | 68.7% |
|
|
| 7: Avoid Bank Conflicts (Linearize) | `16213.4` | 69.7% |
|
|
| 8: Avoid Bank Conflicts (Offset) | `16459.2` | 70.8% |
|
|
| 11: Double Buffering | `17278.3` | 74.3% |
|
|
| 6: Vectorized Mem Access | `18237.3` | 78.4% |
|
|
| 9: Autotuning | `19721.0` | 84.8% |
|
|
| 10: Warptiling | `21779.3` | 93.7% |
|
|
| 0: cuBLAS | `23249.6` | 100.0% |
|
|
<!-- benchmark_results -->
|
|
|
|
## Setup
|
|
|
|
1. Install dependencies: CUDA toolkit 12, Python (+ Seaborn), CMake, Ninja. See [environment.yml](environment.yml).
|
|
1. Configure NVCC compilation parameters. Look up your GPUs compute
|
|
capability [here](https://developer.nvidia.com/cuda-gpus). Then configure the `CMakeLists.txt` and change:
|
|
```cmake
|
|
set(CUDA_COMPUTE_CAPABILITY 80)
|
|
```
|
|
1. Build: `mkdir build && cd build && cmake .. && cmake --build .`
|
|
1. Run one of the kernels: `DEVICE=<device_id> ./sgemm <kernel number>`
|
|
1. Profiling via [NVIDIA Nsight Compute](https://developer.nvidia.com/nsight-compute) (ncu): `make profile KERNEL=<kernel number>`
|
|
|
|
Credit goes to [wangzyon/NVIDIA_SGEMM_PRACTICE](https://github.com/wangzyon/NVIDIA_SGEMM_PRACTICE) for the benchmarking setup.
|