* Adding temperature scaling on Joiner logits:
- T hard-coded to 2.0
- so far best result NCE 0.122 (still not so high)
- the BPE scores were rescaled with 0.2 (but then also incorrect words
get high confidence, visually reasonable histograms are for 0.5 scale)
- BPE->WORD score merging done by min(.) function
(tried also prob-product, and also arithmetic, geometric, harmonic mean)
- without temperature scaling (i.e. scale 1.0), the best NCE was 0.032 (here product merging was best)
Results seem consistent with: https://arxiv.org/abs/2110.15222
Everything tuned on a very-small set of 100 sentences with 813 words and 10.2% WER, a Czech model.
I also experimented with blank posteriors mixed into the BPE confidences,
but no NCE improvement found, so not pushing that.
Temperature scling added also to the Greedy search confidences.
* making `temperature_scale` configurable from outside
* add export of per-token scores (ys, lm, context)
- for best path of the modified-beam-search decoding of transducer
* refactoring JSON export of OnlineRecognitionResult, extending pybind11 API of OnlineRecognitionResult
* export per-token scores also for greedy-search (online-transducer)
- export un-scaled lm_probs (modified-beam search, online-transducer)
- polishing
* fill lm_probs/context_scores only if LM/ContextGraph is present (make Result smaller)
* various fixes to ContextGraph to support open vocabulary keywords decoder
* Add keyword spotter runtime
* Add binary
* First version works
* Minor fixes
* update text2token
* default values
* Add jni for kws
* add kws android project
* Minor fixes
* Remove unused interface
* Minor fixes
* Add workflow
* handle extra info in texts
* Minor fixes
* Add more comments
* Fix ci
* fix cpp style
* Add input box in android demo so that users can specify their keywords
* Fix cpp style
* Fix comments
* Minor fixes
* Minor fixes
* minor fixes
* Minor fixes
* Minor fixes
* Add CI
* Fix code style
* cpplint
* Fix comments
* Fix error
* share GetHypsRowSplits interface and fix getting Topk not taking logprob
* fix lm score of lm fusion and make padding len same with 'icefall/egs/librispeech/ASR/pruned_transducer_stateless7_streaming/decode.py'
* Don't reset model state and feature extractor on endpointing
* support passing decoding_method from commandline
* Add modified_beam_search to Python API
* fix C API example
* Fix style issues
