init v0.23.0

Signed-off-by: Sun Ruoxi <sunruoxi@4paradigm.com>
This commit is contained in:
2026-08-27 15:11:51 +08:00
parent b582a8e7d1
commit 7f8a1b1f7a
2849 changed files with 712887 additions and 22001 deletions

View File

@@ -1,30 +1,33 @@
# {{ model_name }}
- **vLLM Version**: vLLM: {{ vllm_version }} ([{{ vllm_commit[:7] }}](https://github.com/vllm-project/vllm/commit/{{ vllm_commit }})), **vLLM Ascend Version**: {{ vllm_ascend_version }} ([{{ vllm_ascend_commit[:7] }}](https://github.com/vllm-project/vllm-ascend/commit/{{ vllm_ascend_commit }}))
- **Software Environment**: **CANN**: {{ cann_version }}, **PyTorch**: {{ torch_version }}, **torch-npu**: {{ torch_npu_version }}
- **Software Environment**: **CANN**: {{ cann_version }}, **PyTorch**: {{ torch_version }}, **TorchNPU**: {{ torch_npu_version }}
- **Hardware Environment**: {{ hardware }}
- **Parallel mode**: {{ parallel_mode }}
- **Execution mode**: {{ execution_model }}
{% if show_command is not defined or show_command %}
**Command**:
```bash
export MODEL_ARGS={{ model_args }}
lm_eval --model {{ model_type }} --model_args $MODEL_ARGS --tasks {{ datasets }} \
{% if apply_chat_template is defined and (apply_chat_template|string|lower in ["true", "1"]) -%}
lm_eval --model {{ model_type }} --model_args $MODEL_ARGS \
--tasks {{ datasets }} \
{%- if apply_chat_template is defined and (apply_chat_template|string|lower in ["true", "1"]) %}
--apply_chat_template \
{%- endif %}
{% if fewshot_as_multiturn is defined and (fewshot_as_multiturn|string|lower in ["true", "1"]) -%}
{%- if fewshot_as_multiturn is defined and (fewshot_as_multiturn|string|lower in ["true", "1"]) %}
--fewshot_as_multiturn \
{%- endif %}
{% if num_fewshot is defined and num_fewshot != "N/A" -%}
{%- if num_fewshot is defined and num_fewshot != "N/A" %}
--num_fewshot {{ num_fewshot }} \
{%- endif %}
{% if limit is defined and limit != "N/A" -%}
{%- if limit is defined and limit != "N/A" %}
--limit {{ limit }} \
{%- endif %}
--batch_size {{ batch_size }}
--batch_size {{ batch_size }}
```
{% endif %}
| Task | Metric | Value | Stderr |
|-----------------------|-------------|----------:|-------:|