whx feb6bdb12e [Platform][Model Runner] Add hash of request_ids; Change blocksize back to 128. (#293) ...

This PR changes the initial value of blocksize back to 128 and adds hash
value of request id list in model runner for implementing sampling param
cache in sampler.

Signed-off-by: hw_whx <wanghexiang7@huawei.com>
Co-authored-by: hw_whx <wanghexiang7@huawei.com>

2025-03-11 18:50:28 +08:00

..

__init__.py

[Core] Support pooling (#229)

2025-03-04 15:59:34 +08:00

model_runner.py

[Platform][Model Runner] Add hash of request_ids; Change blocksize back to 128. (#293)

2025-03-11 18:50:28 +08:00

pooling_model_runner.py

[Core] Support pooling (#229)

2025-03-04 15:59:34 +08:00

worker.py

[Performance] Change the shape of kv_cache to avoid view of k_cache and v_cache. (#204)

2025-03-05 10:51:07 +08:00