EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
xc-llm-ascend/docs/source/user_guide/feature_guide/structured_output.md
Shanshan Shen 303c08aec9 [Doc] Update structured output doc with upstream link (#5058) 
### What this PR does / why we need it?

Cherry-pick from main
https://github.com/vllm-project/vllm-ascend/pull/4015.

Currently, the usage of structured output feature in vllm-ascend is
totally the same as that in vllm.

Thus, IMO, it's better to remove this doc directly to avoid some case
that there are some changes in the upstream doc and we don't update our
doc in time, which can be misleading to users.

Signed-off-by: shen-shanshan <467638484@qq.com>
2025-12-16 11:32:53 +08:00

951 B
Raw Permalink Blame History

Structured Output Guide

Overview

What is structured output?

LLMs can be unpredictable when you need output in specific formats. Think of asking a model to generate JSON without guidance, it might produce valid text that breaks JSON specification. Structured Output (also known as Guided Decoding) enables LLMs to generate outputs that follow a desired structure while preserving the non-deterministic nature of the system.

In simple terms, structured decoding gives LLMs a "template" to follow. Users provide a schema that "influences" the model output, ensuring compliance with the desired structure.

structured decoding

Usage in vllm-ascend

Currently, the usage of structured output feature in vllm-ascend is totally the same as that in vllm.

Find more examples and explanations about these usages in vLLM official document.