EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

[ci feature] add ci monitor (#10872)

Browse Source
This commit is contained in:
Xiaoyu Zhang
2025-09-25 14:16:29 +08:00
committed by GitHub
parent 3e43eb137b
commit c1f39013b7
3 changed files with 882 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

479
scripts/ci_monitor/README.md Normal file
View File

@@ -0,0 +1,479 @@
# SGLang CI Monitor
A simple tool to analyze CI failures for the SGLang project. This tool fetches recent CI run data from GitHub Actions and provides detailed analysis of failure patterns.
## Features
- **Simple Analysis**: Analyze recent CI runs and identify failure patterns
- **Category Classification**: Automatically categorize failures by type (unit-test, performance, etc.)
- **Pattern Recognition**: Identify common failure patterns (timeouts, build failures, etc.)
- **CI Links**: Direct links to recent failed CI runs for detailed investigation
- **Last Success Tracking**: Track the last successful run for each failed job with PR information
- **JSON Export**: Export detailed analysis data to JSON format
- **Automated Monitoring**: GitHub Actions workflow for continuous CI monitoring
## Installation
No additional dependencies required beyond Python standard library and `requests`:
```bash
pip install requests
```
## Usage
### Basic Usage
```bash
# Replace YOUR_GITHUB_TOKEN with your actual token from https://github.com/settings/tokens
python ci_analyzer.py --token YOUR_GITHUB_TOKEN
```
### Advanced Usage
```bash
# Analyze last 1000 runs
python ci_analyzer.py --token YOUR_GITHUB_TOKEN --limit 1000
# Custom output file
python ci_analyzer.py --token YOUR_GITHUB_TOKEN --limit 500 --output my_analysis.json
```
**Important**: Make sure your GitHub token has `repo` and `workflow` permissions, otherwise you'll get 404 errors.
## Parameters
| Parameter | Default | Description |
|-----------|---------|-------------|
| `--token` | Required | GitHub Personal Access Token |
| `--limit` | 100 | Number of CI runs to analyze |
| `--output` | ci_analysis.json | Output JSON file for detailed data |
## Getting GitHub Token
1. Go to [GitHub Settings > Personal Access Tokens](https://github.com/settings/tokens)
2. Click "Generate new token" > "Generate new token (classic)"
3. **Important**: Select the following permissions:
- `repo` (Full control of private repositories) - **Required for accessing repository data**
- `workflow` (Update GitHub Action workflows) - **Required for reading CI/CD data**
4. Copy the generated token and use it as `YOUR_GITHUB_TOKEN`
**Note**: Without the `repo` and `workflow` permissions, the tool will not be able to access CI run data and will return 404 errors.
## Output
The tool provides:
### Console Output
- Overall statistics (total runs, success rate, etc.)
- Category failure breakdown
- Most frequently failed jobs (Top 50) with direct CI links
- Failure pattern analysis
### JSON Export
Detailed analysis data including:
- Complete failure statistics
- Job failure counts
- Workflow failure counts
- Failure patterns
- Recent failure details
## Example Output
```
============================================================
SGLang CI Analysis Report
============================================================
Overall Statistics:
Total runs: 1000
Successful: 392
Failed: 187
Cancelled: 181
Skipped: 150
Success rate: 39.2%
Category Failure Statistics:
unit-test: 351 failures
accuracy: 84 failures
performance: 55 failures
deepep: 1 failures
Most Frequently Failed Jobs (Top 50):
1. unit-test-backend-1-gpu-amd-mi35x (linux-mi35x-gpu-1): 32 times
Last Success: Run #28893 (2025-09-24 13:35) by Xiaoze Fan: https://github.com/sgl-project/sglang/actions/runs/17978451434
Recent Failures:
- Run #28958 (2025-09-25 01:51) (PR #1 by Yuhao Yao): https://github.com/sgl-project/sglang/actions/runs/17994520789
- Run #28957 (2025-09-25 01:10) (PR #10883 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993860400
- Run #28956 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826732
2. unit-test-backend-1-gpu-amd (linux-mi300-gpu-1, 3): 31 times
Last Success: Run #28903 (2025-09-24 15:38) by gholmes829: https://github.com/sgl-project/sglang/actions/runs/17981905113
Recent Failures:
- Run #28958 (2025-09-25 01:51) (PR #1 by Yuhao Yao): https://github.com/sgl-project/sglang/actions/runs/17994520789
- Run #28957 (2025-09-25 01:10) (PR #10883 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993860400
- Run #28956 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826732
3. accuracy-test-2-gpu-amd (linux-mi35x-gpu-2): 29 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28958 (2025-09-25 01:51) (PR #1 by Yuhao Yao): https://github.com/sgl-project/sglang/actions/runs/17994520789
- Run #28957 (2025-09-25 01:10) (PR #10883 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993860400
- Run #28956 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826732
4. unit-test-backend-1-gpu-amd (linux-mi300-gpu-1, 5): 23 times
Last Success: Run #28906 (2025-09-24 15:43) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17982029749
Recent Failures:
- Run #28958 (2025-09-25 01:51) (PR #1 by Yuhao Yao): https://github.com/sgl-project/sglang/actions/runs/17994520789
- Run #28956 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826732
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
5. unit-test-backend-1-gpu-amd (linux-mi300-gpu-1, 0): 23 times
Last Success: Run #28893 (2025-09-24 13:35) by Xiaoze Fan: https://github.com/sgl-project/sglang/actions/runs/17978451434
Recent Failures:
- Run #28956 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826732
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
6. unit-test-backend-1-gpu-amd (linux-mi300-gpu-1, 7): 18 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28956 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826732
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
7. unit-test-backend-1-gpu-amd (linux-mi325-gpu-1, 3): 17 times
Last Success: Run #28893 (2025-09-24 13:35) by Xiaoze Fan: https://github.com/sgl-project/sglang/actions/runs/17978451434
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
8. build-test (all): 16 times
Last Success: Run #15748 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435618
Recent Failures:
- Run #15824 (2025-09-25 02:16) by Yuan Luo: https://github.com/sgl-project/sglang/actions/runs/17994892894
- Run #15814 (2025-09-25 00:53) by diwei sun: https://github.com/sgl-project/sglang/actions/runs/17993616261
- Run #15812 (2025-09-25 00:35) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993338746
9. bench-test-2-gpu-amd (linux-mi300-gpu-2): 15 times
Last Success: Run #28893 (2025-09-24 13:35) by Xiaoze Fan: https://github.com/sgl-project/sglang/actions/runs/17978451434
Recent Failures:
- Run #28957 (2025-09-25 01:10) (PR #10883 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993860400
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
10. performance-test-1-gpu-part-2-amd (linux-mi300-gpu-1): 15 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
11. accuracy-test-1-gpu-amd (linux-mi325-gpu-1): 15 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
12. unit-test-backend-8-gpu-amd (linux-mi300-gpu-8): 15 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
13. unit-test-backend-1-gpu-amd (linux-mi300-gpu-1, 1): 14 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
14. unit-test-backend-2-gpu-amd (linux-mi300-gpu-2): 14 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
15. performance-test-1-gpu-part-1-amd (linux-mi325-gpu-1): 13 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
16. unit-test-backend-1-gpu-amd (linux-mi300-gpu-1, 2): 13 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
17. unit-test-backend-1-gpu-amd (linux-mi300-gpu-1, 4): 13 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
18. accuracy-test-2-gpu-amd (linux-mi325-gpu-2): 13 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
19. mla-test-1-gpu-amd (linux-mi325-gpu-1): 13 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
20. accuracy-test-2-gpu-amd (linux-mi300-gpu-2): 13 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
21. accuracy-test-1-gpu-amd (linux-mi300-gpu-1): 12 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
22. performance-test-1-gpu-part-2-amd (linux-mi325-gpu-1): 12 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
23. bench-test-2-gpu-amd (linux-mi325-gpu-2): 11 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28957 (2025-09-25 01:10) (PR #10883 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993860400
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
24. unit-test-sgl-kernel-amd (linux-mi325-gpu-1): 11 times
Last Success: Run #28891 (2025-09-24 12:44) by Yuan Luo: https://github.com/sgl-project/sglang/actions/runs/17977053408
Recent Failures:
- Run #28956 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826732
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
25. performance-test-1-gpu-part-1-amd (linux-mi300-gpu-1): 11 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
26. unit-test-backend-1-gpu-amd (linux-mi300-gpu-1, 6): 11 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
27. unit-test-backend-2-gpu-amd (linux-mi325-gpu-2): 11 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
28. unit-test-backend-1-gpu (9): 10 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34623 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826751
- Run #34617 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619818
- Run #34581 (2025-09-24 19:49) by Yineng Zhang: https://github.com/sgl-project/sglang/actions/runs/17987860976
29. unit-test-backend-1-gpu-amd (linux-mi325-gpu-1, 0): 10 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
30. unit-test-backend-1-gpu-amd (linux-mi325-gpu-1, 1): 10 times
Last Success: Run #28891 (2025-09-24 12:44) by Yuan Luo: https://github.com/sgl-project/sglang/actions/runs/17977053408
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
31. mla-test-1-gpu-amd (linux-mi300-gpu-1): 10 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
32. unit-test-backend-1-gpu (5): 9 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34624 (2025-09-25 01:10) (PR #10883 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993860412
- Run #34617 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619818
- Run #34560 (2025-09-24 17:01) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17983919007
33. unit-test-backend-1-gpu-amd (linux-mi325-gpu-1, 2): 9 times
Last Success: Run #28906 (2025-09-24 15:43) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17982029749
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
34. unit-test-sgl-kernel-amd (linux-mi300-gpu-1): 9 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28952 (2025-09-24 23:57) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992751764
- Run #28951 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619816
35. unit-test-backend-1-gpu-amd (linux-mi325-gpu-1, 4): 7 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28955 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426068
- Run #28953 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178855
- Run #28949 (2025-09-24 23:44) (PR #10372 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992591372
36. unit-test-backend-1-gpu-amd (linux-mi325-gpu-1, 6): 7 times
Last Success: Run #28890 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435645
Recent Failures:
- Run #28950 (2025-09-24 23:45) (PR #1 by Xiaoyu Zhang): https://github.com/sgl-project/sglang/actions/runs/17992598523
- Run #28946 (2025-09-24 23:39) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992521547
- Run #28936 (2025-09-24 21:32) by xiafang: https://github.com/sgl-project/sglang/actions/runs/17990244192
37. vllm-dependency-test: 6 times
Last Success: Run #22949 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435651
Recent Failures:
- Run #23028 (2025-09-25 02:39) by xuyongfei.xyf: https://github.com/sgl-project/sglang/actions/runs/17995251178
- Run #23021 (2025-09-25 02:16) by Yuan Luo: https://github.com/sgl-project/sglang/actions/runs/17994892873
- Run #22993 (2025-09-24 21:32) by xiafang: https://github.com/sgl-project/sglang/actions/runs/17990244213
38. per-commit-4-ascend-npu: 6 times
Last Success: Run #10065 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435703
Recent Failures:
- Run #10138 (2025-09-25 02:17) by wangyi: https://github.com/sgl-project/sglang/actions/runs/17994908950
- Run #10137 (2025-09-25 02:16) by Yuan Luo: https://github.com/sgl-project/sglang/actions/runs/17994892896
- Run #10124 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619819
39. unit-test-backend-2-gpu (0): 6 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34624 (2025-09-25 01:10) (PR #10883 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993860412
- Run #34593 (2025-09-24 21:32) by xiafang: https://github.com/sgl-project/sglang/actions/runs/17990244227
- Run #34576 (2025-09-24 18:46) by eigen: https://github.com/sgl-project/sglang/actions/runs/17986403452
40. unit-test-backend-1-gpu (4): 6 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34623 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826751
- Run #34609 (2025-09-24 23:25) (PR #10853 by Yineng Zhang): https://github.com/sgl-project/sglang/actions/runs/17992311361
- Run #34560 (2025-09-24 17:01) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17983919007
41. run-all-notebooks: 6 times
Last Success: Run #26939 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435610
Recent Failures:
- Run #26988 (2025-09-24 23:25) (PR #10853 by Yineng Zhang): https://github.com/sgl-project/sglang/actions/runs/17992311396
- Run #26982 (2025-09-24 21:32) by xiafang: https://github.com/sgl-project/sglang/actions/runs/17990244193
- Run #26973 (2025-09-24 18:46) by eigen: https://github.com/sgl-project/sglang/actions/runs/17986403458
42. per-commit-2-ascend-npu: 5 times
Last Success: Run #10065 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435703
Recent Failures:
- Run #10135 (2025-09-25 02:16) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17994888152
- Run #10109 (2025-09-24 21:32) by xiafang: https://github.com/sgl-project/sglang/actions/runs/17990244207
- Run #10085 (2025-09-24 16:42) by likesen: https://github.com/sgl-project/sglang/actions/runs/17983486537
43. unit-test-backend-8-gpu (0): 5 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34623 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826751
- Run #34621 (2025-09-25 00:40) by Hubert Lu: https://github.com/sgl-project/sglang/actions/runs/17993426098
- Run #34619 (2025-09-25 00:24) (PR #10372 by BBuf): https://github.com/sgl-project/sglang/actions/runs/17993178853
44. pytest-rust: 5 times
Last Success: Run #1761 (2025-09-24 16:39) by Chang Su: https://github.com/sgl-project/sglang/actions/runs/17983415401
Recent Failures:
- Run #1770 (2025-09-24 21:02) by Simo Lin: https://github.com/sgl-project/sglang/actions/runs/17989538977
- Run #1769 (2025-09-24 20:54) by Simo Lin: https://github.com/sgl-project/sglang/actions/runs/17989380799
- Run #1767 (2025-09-24 20:36) by Ata Fatahi: https://github.com/sgl-project/sglang/actions/runs/17988964074
45. per-commit-16-ascend-a3: 4 times
Last Success: Run #10065 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435703
Recent Failures:
- Run #10138 (2025-09-25 02:17) by wangyi: https://github.com/sgl-project/sglang/actions/runs/17994908950
- Run #10135 (2025-09-25 02:16) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17994888152
- Run #10109 (2025-09-24 21:32) by xiafang: https://github.com/sgl-project/sglang/actions/runs/17990244207
46. unit-test-backend-1-gpu (7): 4 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34624 (2025-09-25 01:10) (PR #10883 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993860412
- Run #34573 (2025-09-24 18:45) by Tejesh Anand: https://github.com/sgl-project/sglang/actions/runs/17986382981
- Run #34565 (2025-09-24 17:35) by YAMY: https://github.com/sgl-project/sglang/actions/runs/17984740528
47. unit-test-backend-2-gpu (1): 4 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34593 (2025-09-24 21:32) by xiafang: https://github.com/sgl-project/sglang/actions/runs/17990244227
- Run #34576 (2025-09-24 18:46) by eigen: https://github.com/sgl-project/sglang/actions/runs/17986403452
- Run #34565 (2025-09-24 17:35) by YAMY: https://github.com/sgl-project/sglang/actions/runs/17984740528
48. per-commit-1-ascend-npu: 3 times
Last Success: Run #10065 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435703
Recent Failures:
- Run #10138 (2025-09-25 02:17) by wangyi: https://github.com/sgl-project/sglang/actions/runs/17994908950
- Run #10109 (2025-09-24 21:32) by xiafang: https://github.com/sgl-project/sglang/actions/runs/17990244207
- Run #10085 (2025-09-24 16:42) by likesen: https://github.com/sgl-project/sglang/actions/runs/17983486537
49. unit-test-backend-1-gpu (1): 3 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34623 (2025-09-25 01:07) (PR #10495 by Lianmin Zheng): https://github.com/sgl-project/sglang/actions/runs/17993826751
- Run #34554 (2025-09-24 16:29) by Yuan Luo: https://github.com/sgl-project/sglang/actions/runs/17983177051
- Run #34548 (2025-09-24 15:38) by gholmes829: https://github.com/sgl-project/sglang/actions/runs/17981905143
50. unit-test-backend-1-gpu (8): 3 times
Last Success: Run #34533 (2025-09-24 12:20) by Yuhong Guo: https://github.com/sgl-project/sglang/actions/runs/17976435636
Recent Failures:
- Run #34617 (2025-09-24 23:47) (PR #10881 by Chang Su): https://github.com/sgl-project/sglang/actions/runs/17992619818
- Run #34581 (2025-09-24 19:49) by Yineng Zhang: https://github.com/sgl-project/sglang/actions/runs/17987860976
- Run #34554 (2025-09-24 16:29) by Yuan Luo: https://github.com/sgl-project/sglang/actions/runs/17983177051
Failure Pattern Analysis:
GPU Related Failure: 223 times
Unit Test Failure: 190 times
Accuracy Test Failure: 84 times
Performance Test Failure: 54 times
Other: 34 times
Dependency Installation Failure: 19 times
Build Failure: 15 times
```
## CI Job Categories
The tool automatically categorizes CI jobs into:
- **sgl-kernel**: Kernel-related tests (build, unit tests, MLA tests)
- **unit-test**: Unit tests (frontend, backend with different GPU counts)
- **performance**: Performance tests (latency, throughput benchmarks)
- **accuracy**: Accuracy tests (model evaluation)
- **deepep**: DeepEP-related tests
- **b200**: B200 hardware-specific tests
## Failure Patterns
The tool recognizes these failure patterns:
- **Timeout**: Step execution timeout
- **Unit Test Failure**: Unit test execution failures
- **Performance Test Failure**: Performance benchmark failures
- **Accuracy Test Failure**: Model accuracy evaluation failures
- **Build Failure**: Compilation/build process failures
- **Dependency Installation Failure**: Package installation issues
- **GPU Related Failure**: GPU-specific test failures
- **Other**: Unclassified failures
## Troubleshooting
### Common Issues
1. **404 Error**:
- Ensure the repository name is correct (`sgl-project/sglang`)
- **Most common cause**: Missing `repo` or `workflow` permissions in your GitHub token
- Go to [GitHub Settings > Personal Access Tokens](https://github.com/settings/tokens) and regenerate with correct permissions
2. **403 Error**: Check that your GitHub token has the correct permissions (`repo` and `workflow`)
3. **Rate Limiting**: The tool includes built-in delays to avoid API rate limits
4. **Network Issues**: Ensure stable internet connection
### Debug Mode
For detailed API call information, you can modify the code to include logging:
```python
import logging
logging.basicConfig(level=logging.DEBUG)
```
## Automated Monitoring
The CI monitor is also available as a GitHub Actions workflow that runs automatically every 6 hours. The workflow:
- Analyzes the last 500 CI runs
- Generates detailed reports
- Uploads analysis results as artifacts
### Workflow Configuration
The workflow is located at `.github/workflows/ci-monitor.yml` and uses the `GH_PAT_FOR_NIGHTLY_CI` secret for GitHub API access.
### Manual Trigger
You can manually trigger the workflow from the GitHub Actions tab with custom parameters:
- `limit`: Number of CI runs to analyze (default: 500)
## License
This tool follows the same license as the SGLang project.

389
scripts/ci_monitor/ci_analyzer.py Executable file
View File

@@ -0,0 +1,389 @@
#!/usr/bin/env python3
"""
SGLang CI Analyzer
Simple tool to analyze CI failures for SGLang project
"""
import argparse
import json
import os
import sys
import time
from collections import Counter, defaultdict
from datetime import datetime
from typing import Dict, List
import requests
class SGLangCIAnalyzer:
"""SGLang CI Analyzer"""
def __init__(self, token: str):
self.token = token
self.base_url = "https://api.github.com"
self.repo = "sgl-project/sglang"
self.headers = {
"Authorization": f"token {token}",
"Accept": "application/vnd.github.v3+json",
"User-Agent": "SGLang-CI-Analyzer/1.0",
}
self.session = requests.Session()
self.session.headers.update(self.headers)
def get_recent_runs(self, limit: int = 100) -> List[Dict]:
"""Get recent CI run data"""
print(f"Fetching {limit} recent CI runs...")
all_runs = []
page = 1
per_page = 100
while len(all_runs) < limit:
url = f"{self.base_url}/repos/{self.repo}/actions/runs"
params = {"per_page": min(per_page, limit - len(all_runs)), "page": page}
try:
response = self.session.get(url, params=params)
response.raise_for_status()
data = response.json()
if not data.get("workflow_runs"):
break
all_runs.extend(data["workflow_runs"])
print(f"Fetched {len(all_runs)} runs so far...")
if len(data["workflow_runs"]) < per_page:
break
page += 1
time.sleep(0.1) # Avoid API rate limits
except requests.exceptions.RequestException as e:
print(f"Error fetching CI data: {e}")
break
return all_runs[:limit]
def analyze_ci_failures(self, runs: List[Dict]) -> Dict:
"""Analyze CI failure patterns"""
print("Analyzing CI failure data...")
# SGLang specific job categories
job_categories = {
"sgl-kernel": [
"sgl-kernel-build-wheels",
"sgl-kernel-unit-test",
"sgl-kernel-mla-test",
],
"unit-test": [
"unit-test-frontend",
"unit-test-backend-1-gpu",
"unit-test-backend-2-gpu",
"unit-test-backend-4-gpu",
"unit-test-backend-8-gpu",
],
"performance": [
"performance-test-1-gpu-part-1",
"performance-test-1-gpu-part-2",
"performance-test-2-gpu",
],
"accuracy": ["accuracy-test-1-gpu", "accuracy-test-2-gpu"],
"deepep": ["unit-test-deepep-4-gpu", "unit-test-deepep-8-gpu"],
"b200": ["unit-test-backend-4-gpu-b200"],
}
stats = {
"total_runs": len(runs),
"failed_runs": 0,
"successful_runs": 0,
"cancelled_runs": 0,
"skipped_runs": 0,
"category_failures": defaultdict(int),
"job_failures": defaultdict(int),
"failure_patterns": defaultdict(int),
"job_failure_links": defaultdict(
list
), # Store recent failure links for each job
"job_last_success": {}, # Store last successful run for each job
}
total_runs = len(runs)
for i, run in enumerate(runs, 1):
# Show progress every 10% or every 50 runs, whichever is smaller
if i % max(1, min(50, total_runs // 10)) == 0 or i == total_runs:
progress = (i / total_runs) * 100
print(f"Progress: {i}/{total_runs} ({progress:.1f}%)")
run_status = run.get("conclusion", "unknown")
workflow_name = run.get("name", "Unknown")
run_id = run.get("id")
run_number = run.get("run_number")
created_at = run.get("created_at")
# Count run status
if run_status == "failure":
stats["failed_runs"] += 1
elif run_status == "success":
stats["successful_runs"] += 1
elif run_status == "cancelled":
stats["cancelled_runs"] += 1
elif run_status == "skipped":
stats["skipped_runs"] += 1
# Get detailed job information for all runs
jobs = self._get_job_details(run_id)
run_url = f"https://github.com/{self.repo}/actions/runs/{run_id}"
pr_info = self._get_pr_info(run)
for job in jobs:
job_name = job.get("name", "Unknown")
job_conclusion = job.get("conclusion", "unknown")
# Filter out non-specific CI jobs
if job_name not in [
"check-changes",
"pr-test-finish",
"pr-test-h20-finish",
"lint",
]:
# Record successful jobs (update last success)
if job_conclusion == "success":
stats["job_last_success"][job_name] = {
"url": run_url,
"run_number": run_number,
"created_at": created_at,
"pr_info": pr_info,
}
# Record failed jobs
elif job_conclusion == "failure" and run_status == "failure":
stats["job_failures"][job_name] += 1
# Store failure link (keep only last 3 for each job)
if len(stats["job_failure_links"][job_name]) < 3:
stats["job_failure_links"][job_name].append(
{
"url": run_url,
"run_number": run_number,
"created_at": created_at,
"pr_info": pr_info,
}
)
# Categorize failed jobs
for category, jobs_list in job_categories.items():
if any(
job_pattern in job_name for job_pattern in jobs_list
):
stats["category_failures"][category] += 1
break
# Analyze failure patterns
self._analyze_failure_pattern(job, stats)
time.sleep(0.1) # Avoid API rate limits
return stats
def _get_job_details(self, run_id: int) -> List[Dict]:
"""Get job details for a specific run"""
url = f"{self.base_url}/repos/{self.repo}/actions/runs/{run_id}/jobs"
try:
response = self.session.get(url)
response.raise_for_status()
return response.json().get("jobs", [])
except:
return []
def _get_pr_info(self, run: Dict) -> Dict:
"""Get PR information from a run"""
pr_info = {
"pr_number": None,
"author": run.get("head_commit", {})
.get("author", {})
.get("name", "Unknown"),
"head_sha": run.get("head_sha", ""),
"head_branch": run.get("head_branch", ""),
}
# Try to extract PR number from pull_requests
pull_requests = run.get("pull_requests", [])
if pull_requests:
pr_info["pr_number"] = pull_requests[0].get("number")
return pr_info
def _analyze_failure_pattern(self, job: Dict, stats: Dict):
"""Analyze failure patterns"""
job_name = job.get("name", "")
steps = job.get("steps", [])
for step in steps:
if step.get("conclusion") == "failure":
step_name = step.get("name", "")
# SGLang specific failure pattern recognition
if "timeout" in step_name.lower():
stats["failure_patterns"]["Timeout"] += 1
elif "test" in step_name.lower() and "unit" in job_name.lower():
stats["failure_patterns"]["Unit Test Failure"] += 1
elif "performance" in job_name.lower():
stats["failure_patterns"]["Performance Test Failure"] += 1
elif "accuracy" in job_name.lower():
stats["failure_patterns"]["Accuracy Test Failure"] += 1
elif "build" in step_name.lower():
stats["failure_patterns"]["Build Failure"] += 1
elif "install" in step_name.lower():
stats["failure_patterns"]["Dependency Installation Failure"] += 1
elif "gpu" in job_name.lower():
stats["failure_patterns"]["GPU Related Failure"] += 1
else:
stats["failure_patterns"]["Other"] += 1
def generate_report(self, stats: Dict):
"""Generate CI analysis report"""
print("\n" + "=" * 60)
print("SGLang CI Analysis Report")
print("=" * 60)
# Overall statistics
total = stats["total_runs"]
failed = stats["failed_runs"]
success = stats["successful_runs"]
cancelled = stats["cancelled_runs"]
skipped = stats["skipped_runs"]
success_rate = (success / total * 100) if total > 0 else 0
print(f"\nOverall Statistics:")
print(f" Total runs: {total}")
print(f" Successful: {success}")
print(f" Failed: {failed}")
print(f" Cancelled: {cancelled}")
print(f" Skipped: {skipped}")
print(f" Success rate: {success_rate:.1f}%")
# Category failure statistics
if stats["category_failures"]:
print(f"\nCategory Failure Statistics:")
for category, count in sorted(
stats["category_failures"].items(), key=lambda x: x[1], reverse=True
):
print(f" {category}: {count} failures")
# Most frequently failed jobs with links
if stats["job_failures"]:
print(f"\nMost Frequently Failed Jobs (Top 50):")
for i, (job, count) in enumerate(
sorted(stats["job_failures"].items(), key=lambda x: x[1], reverse=True)[
:50
],
1,
):
print(f" {i:2d}. {job}: {count} times")
# Show last successful run
if job in stats["job_last_success"]:
last_success = stats["job_last_success"][job]
success_date = datetime.fromisoformat(
last_success["created_at"].replace("Z", "+00:00")
)
pr_info = last_success["pr_info"]
pr_text = ""
if pr_info["pr_number"]:
pr_text = (
f" (PR #{pr_info['pr_number']} by {pr_info['author']})"
)
else:
pr_text = f" by {pr_info['author']}"
print(
f" Last Success: Run #{last_success['run_number']} ({success_date.strftime('%Y-%m-%d %H:%M')}){pr_text}: {last_success['url']}"
)
# Show recent failure links
if (
job in stats["job_failure_links"]
and stats["job_failure_links"][job]
):
print(" Recent Failures:")
for link_info in stats["job_failure_links"][job]:
created_at = datetime.fromisoformat(
link_info["created_at"].replace("Z", "+00:00")
)
# Format PR info for failures
pr_info = link_info.get("pr_info", {})
pr_text = ""
if pr_info.get("pr_number"):
pr_text = f" (PR #{pr_info['pr_number']} by {pr_info.get('author', 'Unknown')})"
else:
pr_text = f" by {pr_info.get('author', 'Unknown')}"
print(
f" - Run #{link_info['run_number']} ({created_at.strftime('%Y-%m-%d %H:%M')}){pr_text}: {link_info['url']}"
)
# Failure pattern analysis
if stats["failure_patterns"]:
print(f"\nFailure Pattern Analysis:")
for pattern, count in sorted(
stats["failure_patterns"].items(), key=lambda x: x[1], reverse=True
):
print(f" {pattern}: {count} times")
print("\n" + "=" * 60)
def save_detailed_report(self, stats: Dict, output_file: str = "ci_analysis.json"):
"""Save detailed report to file"""
with open(output_file, "w", encoding="utf-8") as f:
json.dump(stats, f, ensure_ascii=False, indent=2)
print(f"\nDetailed report saved to: {output_file}")
def main():
parser = argparse.ArgumentParser(description="SGLang CI Analyzer")
parser.add_argument("--token", required=True, help="GitHub Personal Access Token")
parser.add_argument(
"--limit",
type=int,
default=100,
help="Number of runs to analyze (default: 100)",
)
parser.add_argument(
"--output",
default="ci_analysis.json",
help="Output file (default: ci_analysis.json)",
)
args = parser.parse_args()
# Create analyzer
analyzer = SGLangCIAnalyzer(args.token)
try:
# Get CI run data
runs = analyzer.get_recent_runs(args.limit)
if not runs:
print("No CI run data found")
return
# Analyze failures
stats = analyzer.analyze_ci_failures(runs)
# Generate report
analyzer.generate_report(stats)
# Save detailed report
analyzer.save_detailed_report(stats, args.output)
except Exception as e:
print(f"Error during analysis: {e}")
sys.exit(1)
if __name__ == "__main__":
main()

14
scripts/ci_monitor/example.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/bash
# Example usage of SGLang CI Analyzer
# IMPORTANT: Get your GitHub token from https://github.com/settings/tokens
# Make sure to select 'repo' and 'workflow' permissions!
# Basic usage - analyze last 100 runs
python3 ci_analyzer.py --token YOUR_GITHUB_TOKEN
# Analyze last 1000 runs
python3 ci_analyzer.py --token YOUR_GITHUB_TOKEN --limit 1000
# Custom output file
python3 ci_analyzer.py --token YOUR_GITHUB_TOKEN --limit 500 --output my_analysis.json