EngineX/xc-llm-kunlun
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[Bugs] Fix Docs Build Problem (#97)

Browse Source 
* [Bugs] Docs fixed

* Update contributing.md

* Update index.md

* fix lua to text

* fix title size
This commit is contained in:
Xinyu Dong
2026-01-10 05:55:40 +08:00
committed by GitHub
parent 8c9cabd760
commit 7be26ca617
17 changed files with 721 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
docs/source/developer_guide/evaluation/accuracy/accuracy_server.md
View File

@@ -1,8 +1,8 @@
## Overall accuracy test
# Overall accuracy test
### EvalScope
## EvalScope
#### 1.Download and install
### 1.Download and install
EvalScope supports use in Python environments. Users can install EvalScope via pip or from source code. Here are examples of both installation methods:
@@ -15,7 +15,7 @@ cd evalscope
pip install -e '.[perf]'
```
#### 2.Dataset preparation script
### 2.Dataset preparation script
```python
from evalscope.collections import CollectionSchema, DatasetInfo, WeightedSampler
@@ -88,20 +88,24 @@ if not os.path.exists(output_dir): # Step 4: Check if the directory exists
# dump the mixed data to a jsonl file
dump_jsonl_data(mixed_data, output_path) # Step 6: Securely write to the file
```
Dataset composition visualization:
```
┌───────────────────────────────────────┐
│ VL-Test (1000 samples) │
├─────────────────┬─────────────────────┤
│ PureText │ Vision │
│ (333 样本) │ (667 样本) │
│ (333 samples) │ (667 samples) │
├─────────────────┼─────────────────────┤
│ • mmlu_pro │ • math_vista │
│ • ifeval │ • mmmu_pro │
│ • gsm8k │ │
└─────────────────┴─────────────────────┘
```
#### 3.Test
### 3.Test
```python
from dotenv import dotenv_values
@@ -134,13 +138,14 @@ task_cfg = TaskConfig(
run_task(task_cfg=task_cfg)
```
Parameter Tuning Guide:
| Parameter | Current value | Effect | Adjustment suggestions |
| ----------------- | ------ | --------------- | ----------------------- |
| `temperature` | 0.6 | Control output diversity | Math problems ↓ 0.3 / Creative writing ↑ 0.9 |
| `top_p` | 0.95 | Filtering low-probability tokens | Reduce "nonsense" |
| `eval_batch_size` | 5 | Number of requests processed in parallel | With sufficient video memory, it can be increased to 10. |
| Parameter | Current value | Effect | Adjustment suggestions |
| ----------------- | ------------- | ---------------------------------------- | -------------------------------------------------------- |
| `temperature` | 0.6 | Control output diversity | Math problems ↓ 0.3 / Creative writing ↑ 0.9 |
| `top_p` | 0.95 | Filtering low-probability tokens | Reduce "nonsense" |
| `eval_batch_size` | 5 | Number of requests processed in parallel | With sufficient video memory, it can be increased to 10. |
Run the test:
@@ -167,20 +172,22 @@ python accuracy.py 2>&1 | tee "$LOG_FILE"
# ========================================
EXIT_CODE=${PIPESTATUS[0]}
if [ $EXIT_CODE -eq 0 ]; then
echo "✅ 评测完成! 日志已保存到: $LOG_FILE"
echo "✅ Evaluation completed! Log saved to: $LOG_FILE"
else
echo "❌ 评测失败! 退出码: $EXIT_CODE 请查看日志: $LOG_FILE"
echo "❌ Evaluation failed! Exit code: $EXIT_CODE Please check the log: $LOG_FILE"
fi
```
#### 4.Common problem fixes
##### 4.1 NLTK resource missing fix
### 4.Common problem fixes
#### 4.1 NLTK resource missing fix
```bash
Resource punkt_tab not found.
```
Solution：
```python
import nltk
import os
@@ -193,13 +200,13 @@ os.makedirs(download_dir, exist_ok=True)
nltk.data.path.append(download_dir)
# Step 3: Download necessary resources
print("🔽 开始下载punkt_tab资源...")
print("🔽 Start downloading punkt_tab resource...")
try:
nltk.download("punkt_tab", download_dir=download_dir)
print("✅ 下载成功!")
print("✅ Download successful!")
except Exception as e:
print(f"❌ 下载失败: {e}")
print("💡 备选方案:手动从GitHub下载")
print(f"❌ Download failed: {e}")
print("💡 Alternative: Download manually from GitHub")
print(
" URL: https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/packages/tokenizers/punkt_tab.zip"
)
@@ -218,7 +225,7 @@ python fix_nltk.py
bash run_accuracy_test.sh
```
#### 5.Results Display
### 5.Results Display
```bash
+-------------+---------------------+--------------+---------------+-------+