初始化项目,由ModelHub XC社区提供模型
Model: Nanthasit/sakthai-plus-1.5b Source: Original Platform
This commit is contained in:
87
.eval_results/benchmark-20260731_043634.yaml
Normal file
87
.eval_results/benchmark-20260731_043634.yaml
Normal file
@@ -0,0 +1,87 @@
|
||||
model: Nanthasit/sakthai-plus-1.5b
|
||||
benchmark_ts: '2026-07-31T04:36:34Z'
|
||||
backend: llama.cpp-gguf-q4_k_m
|
||||
quantization: q4_k_m
|
||||
prompt_type: tool_calling_send_email
|
||||
prompt_length_chars: 1290
|
||||
prompt: '<|im_start|>system
|
||||
|
||||
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.
|
||||
|
||||
|
||||
# Tools
|
||||
|
||||
|
||||
Yo...'
|
||||
trials: 3
|
||||
total_time_s: 47.56
|
||||
input_tokens: 308
|
||||
avg_generation_tps: 20.7
|
||||
has_tool_call_any: true
|
||||
has_tool_call_all: true
|
||||
has_valid_json_any: true
|
||||
has_valid_json_all: true
|
||||
has_correct_answer_any: true
|
||||
has_correct_answer_all: true
|
||||
trials_detail:
|
||||
- seed: 7
|
||||
output_tokens: 63
|
||||
output_length: 231
|
||||
generation_tps: 20.9
|
||||
prompt_tps: 67.0
|
||||
has_tool_call: true
|
||||
has_valid_json: true
|
||||
has_correct_answer: true
|
||||
tool_name: send_email
|
||||
tool_args:
|
||||
to: Beer@Nanthasit.com
|
||||
subject: Plus 1.5B status
|
||||
body: Hi Beer, I wanted to check the benchmark results for the SakThai Plus model.
|
||||
Thanks!
|
||||
response_preview: '<tool_call>
|
||||
|
||||
{"name": "send_email", "arguments": "{\"to\": \"Beer@Nanthasit.com\", \"subject\":
|
||||
\"Plus 1.5B status\", \"body\": \"Hi Beer, I wanted to check the benchmark results
|
||||
for the SakThai Plus m'
|
||||
- seed: 42
|
||||
output_tokens: 65
|
||||
output_length: 243
|
||||
generation_tps: 20.8
|
||||
prompt_tps: 71.0
|
||||
has_tool_call: true
|
||||
has_valid_json: true
|
||||
has_correct_answer: true
|
||||
tool_name: send_email
|
||||
tool_args:
|
||||
to: Beer@Nanthasit.com
|
||||
subject: Plus 1.5B status
|
||||
body: Hi Beer, I wanted to check the results for the SakThai Plus model. Everything
|
||||
looks good so far.
|
||||
response_preview: '<tool_call>
|
||||
|
||||
{"name": "send_email", "arguments": "{\"to\": \"Beer@Nanthasit.com\", \"subject\":
|
||||
\"Plus 1.5B status\", \"body\": \"Hi Beer, I wanted to check the results for the
|
||||
SakThai Plus model. Ever'
|
||||
- seed: 1337
|
||||
output_tokens: 66
|
||||
output_length: 253
|
||||
generation_tps: 20.4
|
||||
prompt_tps: 70.5
|
||||
has_tool_call: true
|
||||
has_valid_json: true
|
||||
has_correct_answer: true
|
||||
tool_name: send_email
|
||||
tool_args:
|
||||
to: Beer@Nanthasit.com
|
||||
subject: Plus 1.5B status
|
||||
body: Hi Beer, I wanted to check the benchmark results for the SakThai Plus model.
|
||||
Everything looks good so far.
|
||||
response_preview: '<tool_call>
|
||||
|
||||
{"name": "send_email", "arguments": "{\"to\": \"Beer@Nanthasit.com\", \"subject\":
|
||||
\"Plus 1.5B status\", \"body\": \"Hi Beer, I wanted to check the benchmark results
|
||||
for the SakThai Plus m'
|
||||
device: cpu
|
||||
threads: 2
|
||||
router_probe_status: 400
|
||||
router_probe_error: Model not supported by provider hf-inference
|
||||
Reference in New Issue
Block a user