EngineX-Iluvatar/enginex-mr_series-sherpa-onnx
Archived
16
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Provide models for mobile-only platforms by fixing batch size to 1 (#1276)

Browse Source
This commit is contained in:
Fangjun Kuang
2024-08-22 19:36:24 +08:00
committed by GitHub
parent d8001d6edc
commit 0e0d04a97a
6 changed files with 287 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
scripts/mobile-asr-models/README.md Normal file
View File

@@ -0,0 +1,18 @@
# Introduction
This folder contains scripts to convert ASR models for mobile platforms
supporting only batch size equal to 1.
The advantage of fixing the batch size to 1 is that it provides more
opportunities for model optimization and quantization.
To give you a concrete example, for the following model
https://k2-fsa.github.io/sherpa/onnx/pretrained_models/online-transducer/zipformer-transducer-models.html#csukuangfj-sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20-bilingual-chinese-english
| | encoder-epoch-99-avg-1.onnx | encoder-epoch-99-avg-1.int8.onnx|
|---|---|---|
|Dynamic batch size| 315 MB| 174 MB|
|Batch size fixed to 1| 242 MB | 100 MB |
The following [colab notebook](https://colab.research.google.com/drive/1RsVZbsxbPjazeGrNNbZNjXCYbEG2F2DU?usp=sharing)
provides examples to use the above two models.