From 5010e0d2ca87716c872b6c78c0c754128812bd90 Mon Sep 17 00:00:00 2001 From: HAI Date: Tue, 29 Oct 2024 10:51:02 -0700 Subject: [PATCH] [3rdparty, document] Add 3rdparty/amd, with profiling and tuning instructions to be added (#1822) --- 3rdparty/amd/profiling/PROFILING.md | 10 ++++++++++ 3rdparty/amd/tuning/TUNING.md | 13 +++++++++++++ 2 files changed, 23 insertions(+) create mode 100644 3rdparty/amd/profiling/PROFILING.md create mode 100644 3rdparty/amd/tuning/TUNING.md diff --git a/3rdparty/amd/profiling/PROFILING.md b/3rdparty/amd/profiling/PROFILING.md new file mode 100644 index 000000000..de0c2ef71 --- /dev/null +++ b/3rdparty/amd/profiling/PROFILING.md @@ -0,0 +1,10 @@ +## Profiling SGLang Infer System with AMD GPUs +This AppNote describes the SGLang profiling technical, code augment and running steps for systems with AMD Instinct GPUs, nevertheless the same procedure may work with Nvidia GPUs too. +Examples and steps are provided in detail, to facilitate easy reproduce and use to localize performance problem towards optimizations. +Two primary methods are covered: +- [RPD](https://github.com/ROCm/rocmProfileData.git) + + +- [Torch Profiler](https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html) + + diff --git a/3rdparty/amd/tuning/TUNING.md b/3rdparty/amd/tuning/TUNING.md new file mode 100644 index 000000000..00f995ebb --- /dev/null +++ b/3rdparty/amd/tuning/TUNING.md @@ -0,0 +1,13 @@ +## Tuning SGLang Infer System with AMD GPUs +This AppNote describes the SGLang performance tuning technical, code harness and running steps for systems with AMD Instinct GPUs. +Harness code, examples and steps are provided in detail, to facilitate easy reproduce & use to tune performance towards workloads. +Three primary runtime areas are covered: +- Triton Kernels + + +- Torch Tunable Ops + + +- Torch Compile + +