Change description of nvidia jetson docs (#3761)
This commit is contained in:
committed by
GitHub
parent
1df6eabd5d
commit
d8d75d256a
@@ -7,18 +7,27 @@ Before starting, ensure the following:
|
|||||||
- [**NVIDIA Jetson AGX Orin Devkit**](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) is set up with **JetPack 6.1** or later.
|
- [**NVIDIA Jetson AGX Orin Devkit**](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) is set up with **JetPack 6.1** or later.
|
||||||
- **CUDA Toolkit** and **cuDNN** are installed.
|
- **CUDA Toolkit** and **cuDNN** are installed.
|
||||||
- Verify that the Jetson AGX Orin is in **high-performance mode**:
|
- Verify that the Jetson AGX Orin is in **high-performance mode**:
|
||||||
```bash
|
```bash
|
||||||
sudo nvpmodel -m 0
|
sudo nvpmodel -m 0
|
||||||
```
|
```
|
||||||
- A custom PyPI index hosted at https://pypi.jetson-ai-lab.dev/jp6/cu126, tailored for NVIDIA Jetson Orin platforms and CUDA 12.6.
|
|
||||||
|
|
||||||
To install torch from this index:
|
|
||||||
```bash
|
|
||||||
pip install torch --index-url https://pypi.jetson-ai-lab.dev/jp6/cu126
|
|
||||||
```
|
|
||||||
* * * * *
|
* * * * *
|
||||||
## Installation
|
## Installing and running SGLang with Jetson Containers
|
||||||
Please refer to [Installation Guide](https://docs.sglang.ai/start/install.html) to install FlashInfer and SGLang.
|
Clone the jetson-containers github repository:
|
||||||
|
```
|
||||||
|
git clone https://github.com/dusty-nv/jetson-containers.git
|
||||||
|
```
|
||||||
|
Run the installation script:
|
||||||
|
```
|
||||||
|
bash jetson-containers/install.sh
|
||||||
|
```
|
||||||
|
Build the container:
|
||||||
|
```
|
||||||
|
CUDA_VERSION=12.6 jetson-containers build sglang
|
||||||
|
```
|
||||||
|
Run the container:
|
||||||
|
```
|
||||||
|
docker run --runtime nvidia -it --rm --network=host IMAGE_NAME
|
||||||
|
```
|
||||||
* * * * *
|
* * * * *
|
||||||
|
|
||||||
Running Inference
|
Running Inference
|
||||||
|
|||||||
Reference in New Issue
Block a user