FP4 weight loading and inference (2/2) (#3972)

This commit is contained in:
Trevor Morris
2025-04-08 17:26:21 -07:00
committed by GitHub
parent 5039d54772
commit 11d760d56a
6 changed files with 262 additions and 1 deletions

View File

@@ -156,6 +156,14 @@ unset CCACHE_READONLY
python -m uv build --wheel -Cbuild-dir=build --color=always .
```
##### Configuring CMake Build Options
Cmake options can be configuring by adding `-Ccmake.define.<option>=<value>` to the `uv build` flags.
For example, to enable building FP4 kernels, use:
```bash
python -m uv build --wheel -Cbuild-dir=build -Ccmake.define.SGL_KERNEL_ENABLE_FP4=1 --color=always .
```
See CMakeLists.txt for more options.
### Testing & Benchmarking
1. Add pytest tests in [tests/](https://github.com/sgl-project/sglang/tree/main/sgl-kernel/tests), if you need to skip some test, please use `@pytest.mark.skipif`