2 Commits

Author	SHA1	Message	Date
whx	0d3463400a	[Performance] Change the shape of kv_cache to avoid view of k_cache and v_cache. (#204) ... This PR changes the shape of kv cache to avoid the view of k_cache and v_cache. What's more, cache the metadata of k_cache and v_cache to avoid duplicative slice operations to improve performance. Signed-off-by: hw_whx <wanghexiang7@huawei.com>	2025-03-05 10:51:07 +08:00
wangxiyuan	5f465010de	[Core] Cherry pick from 0.7.1 to keep the main code newest (#127) ... Cherry pick from 0.7.1 to keep the main code newest Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>	2025-02-21 17:07:37 +08:00