This change ensures that the main audio processing loop is not blocked by
long-running operations in `runSecondPass`, improving responsiveness and
reducing the risk of missing parts of input speech.
* Support using onnxruntime 1.16.0 with CUDA 11.4 on Jetson Orin NX.
The pre-built onnxruntime libs are provided by the community
using the following command:
```bash
./build.sh --build_shared_lib --config Release --update \
--build --parallel --use_cuda \
--cuda_home /usr/local/cuda \
--cudnn_home /usr/lib/aarch64-linux-gnu 2>&1 | tee my-log.txt
```
See also https://github.com/microsoft/onnxruntime/discussions/11226
---
Info about the board:
```
Model: NVIDIA Orin NX T801-16GB - Jetpack 5.1.4 [L4T 35.6.0]
```
```
nvidia@nvidia-desktop:~/Downloads$ head -n 1 /etc/nv_tegra_release
# R35 (release), REVISION: 6.0, GCID: 37391689, BOARD: t186ref, EABI: aarch64, DATE: Wed Aug 28 09:12:27 UTC 2024
nvidia@nvidia-desktop:~/Downloads$ uname -r
5.10.216-tegra
nvidia@nvidia-desktop:~/Downloads$ lsb_release -i -r
Distributor ID: Ubuntu
Release: 20.04
nvidia@nvidia-desktop:~/Downloads$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:43:33_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
nvidia@nvidia-desktop:~/Downloads$ dpkg -l libcudnn8
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-====================-============-=================================
ii libcudnn8 8.6.0.166-1+cuda11.4 arm64 cuDNN runtime libraries
nvidia@nvidia-desktop:~/Downloads$ dpkg -l tensorrt
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-==================-============-=================================
ii tensorrt 8.5.2.2-1+cuda11.4 arm64 Meta package for TensorRT
```
