whx feb6bdb12e [Platform][Model Runner] Add hash of request_ids; Change blocksize back to 128. (#293) ...

This PR changes the initial value of blocksize back to 128 and adds hash
value of request id list in model runner for implementing sampling param
cache in sampler.

Signed-off-by: hw_whx <wanghexiang7@huawei.com>
Co-authored-by: hw_whx <wanghexiang7@huawei.com>

2025-03-11 18:50:28 +08:00

..

models

register qwen2_vl to rewrite qwen2_vl forwad (#241)

2025-03-07 15:41:47 +08:00

ops

[Fix] Remove npu_group_topk before CANN version update (#242)

2025-03-06 09:02:46 +08:00

quantization

[Feature] Modify description and api for ascend quantization (#243)

2025-03-06 15:17:25 +08:00

worker

[Platform][Model Runner] Add hash of request_ids; Change blocksize back to 128. (#293)

2025-03-11 18:50:28 +08:00

__init__.py

register qwen2_vl to rewrite qwen2_vl forwad (#241)

2025-03-07 15:41:47 +08:00

attention.py

[Performance] Change the shape of kv_cache to avoid view of k_cache and v_cache. (#204)

2025-03-05 10:51:07 +08:00

communicator.py

[Dist] Set device as rank (#202)

2025-03-03 09:23:13 +08:00

platform.py

[Platform][Model Runner] Add hash of request_ids; Change blocksize back to 128. (#293)

2025-03-11 18:50:28 +08:00

utils.py

[Worker] Register mindie_turbo while initializing NPUWorker (#13)

2025-02-07 16:47:17 +08:00