Olivier Chafik f5cd27b71d server: streaming of tool calls and thoughts when --jinja is on (#12379) ...

* add common_json w/ support for truncated json healing

* add common_chat_msg_diff

* partial common_chat_parse

* refactor parser w/ optionals

* server: wire chat diffs in stream mode

* fix trigger of thinking models (must happen after thoughts are closed)

* fix functionary v3.2 raw python!

* rename: common_chat_syntax (now contains format)

* rm common_regex.at_start

* don't return empty <think></think>

* accommodate yet another deepseek r1 distill fantasy syntax (`<｜tool▁calls｜>`)

* fix QwQ 32B tool call parsing after thoughts (hermes2)

* better logs for grammar triggers

* consume spaces after parse_json_tool_calls

* fix required tool calls w/ thinking models that have pre-opened thinking tags

* fix thinking model's initial trigger + test qwq's template

* run most test_tool_call tests in stream + non-stream modes

* make functionary v3.2 parsing more strict (differentiate first match from others)

* send final diff from server, to close off raw python arguments

* support partial content streaming in Generic mode

* tool-call: allow content prelude before hermes2 tool calls (for Qwen2.5)

* Update function-calling.md

* Update tool_bench.py

* chat-parser: remove input from exception (llm output may contain PII)

---------

Co-authored-by: ochafik <ochafik@google.com>
Co-authored-by: Olivier Chafik <ochafik@users.noreply.github.com>

2025-05-25 01:48:08 +01:00

..

apple

llama : add xcframework build script (#11996)

2025-03-05 06:30:31 +01:00

build-info.sh

llama : reorganize source code + improve CMake (#8006)

2024-06-26 18:33:02 +03:00

check-requirements.sh

repo : update links to new url (#11886)

2025-02-15 16:40:57 +02:00

ci-run.sh

ci : add model tests + script wrapper (#4586)

2024-01-26 14:18:00 +02:00

compare-commits.sh

scripts : change build path to "build-bench" for compare-commits.sh (#10836)

2024-12-15 18:44:47 +02:00

compare-llama-bench.py

scripts : fix compare-llama-bench.py show parameter (#13514)

2025-05-14 08:41:01 +02:00

debug-test.sh

scripts : fix spelling typo in messages and comments (#9782)

2024-10-08 09:19:53 +03:00

fetch_server_test_models.py

llama : move end-user examples to tools directory (#13249)

2025-05-02 20:27:13 +02:00

gen-authors.sh

license : update copyright notice + add AUTHORS (#6405)

2024-04-09 09:23:19 +03:00

gen-unicode-data.py

py : type-check all Python scripts with Pyright (#8341)

2024-07-07 15:04:39 -04:00

get_chat_template.py

scripts: corrected encoding when getting chat template (#11866) (#11907)

2025-02-18 10:30:16 +01:00

get-flags.mk

build : pass all warning flags to nvcc via -Xcompiler (#5570)

2024-02-18 16:21:52 -05:00

get-hellaswag.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

get-pg.sh

scripts : improve get-pg.sh (#4838)

2024-01-09 19:21:13 +02:00

get-wikitext-2.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

get-wikitext-103.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

get-winogrande.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

hf.sh

scripts : restore hf.sh (#11288)

2025-01-18 13:18:32 +02:00

install-oneapi.bat

support SYCL backend windows build (#5208)

2024-01-31 08:08:07 +05:30

qnt-all.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

run-all-perf.sh

scripts : add pipefail

2023-08-29 10:50:30 +03:00

run-all-ppl.sh

build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)

2024-06-13 00:41:52 +01:00

sync-ggml-am.sh

scripts : fix sync-ggml-am.sh

2025-04-11 00:17:47 +03:00

sync-ggml.last

sync : ggml

2025-05-19 13:29:56 +03:00

sync-ggml.sh

scripts : update sync + fix cmake merge

2025-03-27 10:09:29 +02:00

tool_bench.py

server: streaming of tool calls and thoughts when --jinja is on (#12379)

2025-05-25 01:48:08 +01:00

tool_bench.sh

tool-call: fix Qwen 2.5 Coder support, add micro benchmarks, support trigger patterns for lazy grammars (#12034)

2025-03-05 13:05:13 +00:00

verify-checksum-models.py

convert.py : add python logging instead of print() (#6511)

2024-05-03 22:36:41 +03:00

xxd.cmake

llama : move end-user examples to tools directory (#13249)

2025-05-02 20:27:13 +02:00