Commit Graph

3 Commits

Author SHA1 Message Date
Chayenne
1703d766d8 CI: skip special token for engine token ids unit test (#2648) 2024-12-29 13:52:50 -08:00
Shi Shuai
fad29f7f52 CI: Fix unittest for engine input token ids and output token ids (#2646) 2024-12-29 13:28:59 -08:00
Shi Shuai
35bdb48557 [Feature] Get Token IDs with Engine.generate() (#2636)
Co-authored-by: Chayenne <zhaochen20@outlook.com>
2024-12-29 12:28:27 -08:00