whx 0d3463400a [Performance] Change the shape of kv_cache to avoid view of k_cache and v_cache. (#204) ...

This PR changes the shape of kv cache to avoid the view of k_cache and
v_cache.
What's more, cache the metadata of k_cache and v_cache to avoid
duplicative slice operations to improve performance.

Signed-off-by: hw_whx <wanghexiang7@huawei.com>

2025-03-05 10:51:07 +08:00

..

ops

[CI] Fix unsolved bugs caused by pta api change. (#190)

2025-02-27 19:52:28 +08:00

quantization

[Performance] Change the shape of kv_cache to avoid view of k_cache and v_cache. (#204)

2025-03-05 10:51:07 +08:00

worker

[Performance] Change the shape of kv_cache to avoid view of k_cache and v_cache. (#204)

2025-03-05 10:51:07 +08:00

__init__.py

[Core] Init vllm-ascend (#3)

2025-02-05 10:53:12 +08:00

attention.py

[Performance] Change the shape of kv_cache to avoid view of k_cache and v_cache. (#204)

2025-03-05 10:51:07 +08:00

communicator.py

[Dist] Set device as rank (#202)

2025-03-03 09:23:13 +08:00

platform.py

[Core] Support pooling (#229)

2025-03-04 15:59:34 +08:00

utils.py

[Worker] Register mindie_turbo while initializing NPUWorker (#13)

2025-02-07 16:47:17 +08:00