由于SenseVoiceSmall.from_pretrained()
调用的funasr.auto.auto_model.AutoModel.build_model()默认device是cuda
(在cuda available的环境中)
```py
device = kwargs.get("device", "cuda")
if not torch.cuda.is_available() or kwargs.get("ngpu", 1) == 0:
device = "cpu"
kwargs["batch_size"] = 1
kwargs["device"] = device
```
而export-onnx.py里的tensor默认都是cpu, 导致
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu
所以直接在加载model的时候指定cpu
This change ensures that the main audio processing loop is not blocked by
long-running operations in `runSecondPass`, improving responsiveness and
reducing the risk of missing parts of input speech.
* Support using onnxruntime 1.16.0 with CUDA 11.4 on Jetson Orin NX.
The pre-built onnxruntime libs are provided by the community
using the following command:
```bash
./build.sh --build_shared_lib --config Release --update \
--build --parallel --use_cuda \
--cuda_home /usr/local/cuda \
--cudnn_home /usr/lib/aarch64-linux-gnu 2>&1 | tee my-log.txt
```
See also https://github.com/microsoft/onnxruntime/discussions/11226
---
Info about the board:
```
Model: NVIDIA Orin NX T801-16GB - Jetpack 5.1.4 [L4T 35.6.0]
```
```
nvidia@nvidia-desktop:~/Downloads$ head -n 1 /etc/nv_tegra_release
# R35 (release), REVISION: 6.0, GCID: 37391689, BOARD: t186ref, EABI: aarch64, DATE: Wed Aug 28 09:12:27 UTC 2024
nvidia@nvidia-desktop:~/Downloads$ uname -r
5.10.216-tegra
nvidia@nvidia-desktop:~/Downloads$ lsb_release -i -r
Distributor ID: Ubuntu
Release: 20.04
nvidia@nvidia-desktop:~/Downloads$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:43:33_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
nvidia@nvidia-desktop:~/Downloads$ dpkg -l libcudnn8
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-====================-============-=================================
ii libcudnn8 8.6.0.166-1+cuda11.4 arm64 cuDNN runtime libraries
nvidia@nvidia-desktop:~/Downloads$ dpkg -l tensorrt
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-==================-============-=================================
ii tensorrt 8.5.2.2-1+cuda11.4 arm64 Meta package for TensorRT
```
