EngineX-Iluvatar/enginex-mr_series-sherpa-onnx
Archived
16
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Refactor exporting NeMo models (#2362)

Browse Source 
Refactors and extends model export support to include new NeMo Parakeet TDT int8 variants for English and Japanese, updating the Kotlin API, export scripts, test runners, and CI workflows.

- Added support for two new int8 model types in OfflineRecognizer.kt.
- Enhanced Python export scripts to perform dynamic quantization and metadata injection.
- Updated shell scripts and GitHub workflows to package, test, and publish int8 model artifacts.
This commit is contained in:
Fangjun Kuang
2025-07-09 16:02:12 +08:00
committed by GitHub
parent f1405779cf
commit 6122a678f5
19 changed files with 671 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.github/workflows/export-nemo-fast-conformer-hybrid-transducer-transducer.yaml vendored
View File

@@ -54,13 +54,18 @@ jobs:
curl -SL -O https://hf-mirror.com/csukuangfj/sherpa-onnx-nemo-ctc-en-conformer-small/resolve/main/test_wavs/trans.txt
popd
cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms
cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms
cp -av test_wavs ./sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms
tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms
tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms
tar cjvf sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms.tar.bz2 sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms
models=(
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms-int8
)
for m in ${models[@]}; do
cp -av test_wavs $m
tar cjvf $m.tar.bz2 $m
done
- name: Release
uses: svenstaro/upload-release-action@v2
@@ -71,3 +76,41 @@ jobs:
repo_name: k2-fsa/sherpa-onnx
repo_token: ${{ secrets.UPLOAD_GH_SHERPA_ONNX_TOKEN }}
tag: asr-models
- name: Publish to huggingface
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
uses: nick-fields/retry@v3
with:
max_attempts: 20
timeout_seconds: 200
shell: bash
command: |
git config --global user.email "csukuangfj@gmail.com"
git config --global user.name "Fangjun Kuang"
models=(
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-80ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-480ms-int8
sherpa-onnx-nemo-streaming-fast-conformer-transducer-en-1040ms-int8
)
for m in ${models[@]}; do
rm -rf huggingface
export GIT_LFS_SKIP_SMUDGE=1
export GIT_CLONE_PROTECTION_ACTIVE=false
git clone https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$m huggingface
cp -av $m/* huggingface
cd huggingface
git lfs track "*.onnx"
git lfs track "*.wav"
git status
git add .
git status
git commit -m "first commit"
git push https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$m main
cd ..
done