28 lines
1.1 KiB
Markdown
28 lines
1.1 KiB
Markdown
---
|
|
license: llama3.2
|
|
datasets:
|
|
- teknium/OpenHermes-2.5
|
|
- NousResearch/hermes-function-calling-v1
|
|
base_model:
|
|
- minpeter/QLoRA-Llama-3.2-1B-chatml-tool-v4
|
|
- meta-llama/Llama-3.2-1B
|
|
language:
|
|
- en
|
|
pipeline_tag: text-generation
|
|
library_name: transformers
|
|
tags:
|
|
- axolotl
|
|
- merge
|
|
---
|
|
|
|
|
|
## Model Performance Comparison (BFCL)
|
|
|
|
| task name | minpeter/Llama-3.2-1B-chatml-tool-v4 | meta-llama/Llama-3.2-1B-Instruct (measure) | meta-llama/Llama-3.2-1B-Instruct (Reported) |
|
|
|-----------------|-----------------------------------|-----------------------------------|---------------------------------------|
|
|
| parallel_multiple | 0.000 | 0.025 | **0.15** |
|
|
| parallel | 0.000 | 0.035 | **0.36** |
|
|
| simple | **0.7725** | 0.215 | 0.2925 |
|
|
| multiple | **0.765** | 0.17 | 0.335 |
|
|
|
|
*Parallel calls are not taken into account. 0 points are expected. We plan to fix this in next version.* |