678 B
678 B
External ToolACE Evaluation
- Base:
openbmb/MiniCPM5-1B - Adapter:
./minicpm5_tooluse_nemotron_dpo_from_repair_dpo - Eval dataset:
Team-ACE/ToolACE - Cases:
300
| Metric | Base | Final adapter | Delta |
|---|---|---|---|
| parseable_rate | 0.0133 | 0.9933 | +0.9800 |
| valid_name_rate | 0.0133 | 0.9700 | +0.9567 |
| expected_name_rate | 0.0133 | 0.9267 | +0.9133 |
| args_exact_rate | 0.1500 | 0.6533 | +0.5033 |
| arg_key_overlap | 0.0033 | 0.7517 | +0.7484 |
| no_schema_copy_rate | 1.0000 | 1.0000 | +0.0000 |
| no_repetition_rate | 0.9967 | 1.0000 | +0.0033 |
| stopped_cleanly_rate | 0.0000 | 0.1500 | +0.1500 |
Final adapter beats base on 7/8 metrics.