[FEAT] JSON constrained support (#1125)

Co-authored-by: Yineng Zhang <me@zhyncs.com>
This commit is contained in:
havetc
2024-08-26 18:37:26 +02:00
committed by GitHub
parent c5fe11a8e1
commit 9935f97b3e
10 changed files with 147 additions and 3 deletions

View File

@@ -60,6 +60,9 @@ spaces_between_special_tokens: bool = True,
regex: Optional[str] = None,
# Do parallel sampling and return `n` outputs.
n: int = 1,
# Constrains the output to follow a given JSON schema.
# `regex` and `json_schema` cannot be set at the same time.
json_schema: Optional[str] = None,
## Penalties. See [Performance Implications on Penalties] section below for more informations.