Commit Graph

13 Commits

Author SHA1 Message Date
Karel Vesely
38c072dcb2 Track token scores (#571)
* add export of per-token scores (ys, lm, context)

- for best path of the modified-beam-search decoding of transducer

* refactoring JSON export of OnlineRecognitionResult, extending pybind11 API of OnlineRecognitionResult

* export per-token scores also for greedy-search (online-transducer)

- export un-scaled lm_probs (modified-beam search, online-transducer)
- polishing

* fill lm_probs/context_scores only if LM/ContextGraph is present (make Result smaller)
2024-02-29 06:28:45 +08:00
Karel Vesely
3f2a17ef47 Fixes issue #535 , fix hexa 1-char tokens in ASR output. (#550)
- Avoid output like : `[' K', '<0x64>', '<0x79>', 'ť', ' a', '<0x75>',
  'to', 'bu', '<0x73>', '<0x75>', ... ]` with regular 500 BPE units.
- Don't rewrite 1-char tokens in range [ 0x20 (space) .. 0x7E (tilde) ]
2024-01-26 19:23:20 +08:00
chiiyeh
e7b18a2139 add blank_penalty for online transducer (#548) 2024-01-26 12:12:13 +08:00
Fangjun Kuang
e215d0c39a Fix Byte BPE string results for Python. (#512)
It ignores invalid UTF8 strings.
2024-01-03 16:03:24 +08:00
Fangjun Kuang
98b67ad850 Fix reading hotwords file for android (#354) 2023-10-11 12:20:50 +08:00
Fangjun Kuang
33a5765169 Print a more user-friendly error message when using --hotwords-file. (#344) 2023-09-26 11:04:20 +08:00
Fangjun Kuang
552a267c23 Set is_final and start_time for online websocket server. (#342)
* Set is_final and start_time for online websocket server.

* Convert timestamps to a json array
2023-09-25 15:12:07 +08:00
Wei Kang
47184f9db7 Refactor hotwords,support loading hotwords from file (#296) 2023-09-14 19:33:17 +08:00
Fangjun Kuang
a12ebfab22 treat unk as blank (#299) 2023-09-07 15:12:29 +08:00
Fangjun Kuang
a0a747a0c0 add endpointing for online websocket server (#294) 2023-08-31 14:41:04 +08:00
Fangjun Kuang
6038e2aa62 Support streaming paraformer (#263) 2023-08-14 10:32:14 +08:00
frankyoujian
9dcad7e963 Reinitialize context state after Reset stream when using contexts (#257) 2023-08-10 14:19:40 +08:00
Fangjun Kuang
79c2ce5dd4 Refactor online recognizer (#250)
* Refactor online recognizer.

Make it easier to support other streaming models.

Note that it is a breaking change for the Python API.
`sherpa_onnx.OnlineRecognizer()` used before should be
replaced by `sherpa_onnx.OnlineRecognizer.from_transducer()`.
2023-08-09 20:27:31 +08:00