Commit Graph

8 Commits

Author SHA1 Message Date
Li Wang
b4cb0eecb6 [CI] Hotfix on benchmark results path (#1076)
### What this PR does / why we need it?
Fix benchmark results path

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-06-05 12:53:46 +08:00
Li Wang
31dd471574 [CI] Add workflow_dispatch and use main benchmarks directly (#1071)
### What this PR does / why we need it?

This is for the benchmark iteration, which will change the benchmark
scripts while checkouting each commit. So we need ensure the benchmark
scripts always available.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Manaully

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-06-05 10:29:30 +08:00
Yikun Jiang
9e855b70be Adjust concurrency group for each npu workflow (#1068)
### What this PR does / why we need it?
Adjust concurrency group for each npu workflow
- for pd and benchmarks share the static-08-01, so only one job can runs
on
- other job one PR/schedule should have only 1 job runs

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
2025-06-05 09:17:04 +08:00
Li Wang
517811449e [CI] Re-enable sleep mode test and skip failure breaking CI (#990)
### What this PR does / why we need it?

- Re-enable sleep mode test
- Fix nightly performance benchmark workflow
- Fix model-runner-v1 bug for upstream
[change](https://github.com/vllm-project/vllm/pull/18654)
---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-06-04 16:24:16 +08:00
Li Wang
eb2701e0b2 [CI] Remove workflow_dispatch and change schedule time (#1056)
### What this PR does / why we need it?

- Remove workflow_dispatch 
-  Change schedule time to 2:00 UTC+8
### Does this PR introduce _any_ user-facing change?


### How was this patch tested?
CI passed

---------

Signed-off-by: wangli <858794774@qq.com>
Co-authored-by: wangli <858794774@qq.com>
2025-06-04 01:19:20 +08:00
Li Wang
06fb5a8d81 [CI][Bugfix] Upgrade escli to v0.2.1 to fix benchmark deps (#1055)
### What this PR does / why we need it?

Update escli-tool to v.0.2.1 to fix deps bug

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

Signed-off-by: wangli <858794774@qq.com>
2025-06-04 01:03:56 +08:00
Li Wang
76dacf3fa0 [CI][Benchmark] Optimize performance benchmark workflow (#1039)
### What this PR does / why we need it?

This is a post patch of #1014, for some convenience optimization
- Set cached dataset path for speed
- Use pypi to install escli-tool
- Add benchmark results convert script to have a developer-friendly
result
- Patch the `benchmark_dataset.py` to disable streaming load for
internet
- Add more trigger ways for different purpose, `pr` for debug,
`schedule` for daily test, `dispatch` and `pr-labled` for manual testing
of a single(current) commit
- Disable latency test for `qwen-2.5-vl`, (This script does not support
multi-modal yet)

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-06-03 23:38:34 +08:00
Li Wang
d9fb027068 [CI] Add benchmark workflows (#1014)
### What this PR does / why we need it?

Add benchmark workflows

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Run locally

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-05-30 22:42:44 +08:00