5 Commits

Author	SHA1	Message	Date
Lianmin Zheng	d4fc1a70e3	Crash the server correctly during error (#2231)	2024-11-28 00:22:39 -08:00
Lianmin Zheng	9c939a3d8b	Clean up metrics code (#1972)	2024-11-09 15:43:20 -08:00
Lianmin Zheng	0abbf289a8	Unify the model type checking (#1905)	2024-11-03 12:25:39 -08:00
Lianmin Zheng	86fc0d79d0	Add a watch dog thread (#1816)	2024-10-27 02:00:50 -07:00
havetc	ecb8bad276	Returning a per request metric for number of cached_tokens read (#1599)	2024-10-16 11:49:22 -07:00