3e5ae49160
### What this PR does / why we need it? Update online serving tutorials for `Qwen2-Audio`. Part of https://github.com/vllm-project/vllm-ascend/issues/3508. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: shen-shanshan <467638484@qq.com>
24 lines
928 B
Django/Jinja
24 lines
928 B
Django/Jinja
{% set audio_count = namespace(value=0) %}
|
|
{% for message in messages %}
|
|
{% if loop.first and message['role'] != 'system' %}
|
|
<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n
|
|
{% endif %}
|
|
<|im_start|>{{ message['role'] }}\n
|
|
{% if message['content'] is string %}
|
|
{{ message['content'] }}<|im_end|>\n
|
|
{% else %}
|
|
{% for content in message['content'] %}
|
|
{% if 'audio' in content or 'audio_url' in content or message['type'] == 'audio' or content['type'] == 'audio' %}
|
|
{% set audio_count.value = audio_count.value + 1 %}
|
|
Audio {{ audio_count.value }}: <|audio_bos|><|AUDIO|><|audio_eos|>\n
|
|
{% elif 'text' in content %}
|
|
{{ content['text'] }}
|
|
{% endif %}
|
|
{% endfor %}
|
|
<|im_end|>\n
|
|
{% endif %}
|
|
{% endfor %}
|
|
{% if add_generation_prompt %}
|
|
<|im_start|>assistant\n
|
|
{% endif %}
|