Commit Graph

18 Commits

Author SHA1 Message Date
Byron Hsu
ef995dae1e [router] Health check on worker before adding to the router (#2392) 2024-12-07 15:39:54 -08:00
Byron Hsu
c36736c841 [router] Add remove worker api (#2380) 2024-12-06 17:16:03 -08:00
Byron Hsu
1bf9e34745 [router] add remove tenant method in the radix tree (#2379) 2024-12-06 11:53:15 -08:00
Byron Hsu
499c85f131 [Router] remove duplicate char count (#2378) 2024-12-06 11:26:07 -08:00
Byron Hsu
67b657945a [router] support /add_worker api (#2369) 2024-12-06 01:17:04 -08:00
Byron Hsu
0b46b951ae Fix rust warning (#2208) 2024-11-26 15:00:41 -08:00
Byron Hsu
1f76fc6e3f [router] Rust e2e test (#2184) 2024-11-25 16:02:03 -08:00
Byron Hsu
4d62bca542 [router] Replace print with logger (#2183) 2024-11-25 13:36:02 -08:00
Byron Hsu
4b0a1c9365 Replace prob based with threshold based load balancing (#2170) 2024-11-24 23:17:11 -08:00
Henry Hyeonmok Ko
dbe1729395 Merged three native APIs into one: get_server_info (#2152) 2024-11-24 01:37:58 -08:00
Byron Hsu
7921690893 add prefix match for certain tenant (#2147) 2024-11-23 15:35:02 -08:00
Byron Hsu
bbb81c2457 Add more api routes (completion, health, etc) to the router (#2146) 2024-11-23 15:10:26 -08:00
Byron Hsu
cbedd1db1d [router] cache-aware load-balancing router v1 (#2114) 2024-11-23 08:34:48 -08:00
Byron Hsu
00ffde206f setup router python binding ci (#1999) 2024-11-11 12:19:32 -08:00
Byron Hsu
f9633fa9b9 [rust] cache-aware DP - approx tree (#1934) 2024-11-10 21:57:32 -08:00
Byron Hsu
96766101b4 [rust] refactor server and router (#1922) 2024-11-06 00:02:02 -08:00
Byron Hsu
530ff541cf [router] Impl radix tree and set up CI (#1893)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2024-11-04 10:56:52 -08:00
Byron Hsu
3839be2913 [Router] Add a rust-based router (#1790) 2024-10-28 09:49:48 -07:00