EngineX-Ascend/enginex-ascend-910-llama.cpp
10
0
Fork 0
Code Issues Pull Requests Actions 4 Projects Releases Wiki Activity

vocab : mark EOT token for Granite models (#16499)

Browse Source 
* vocab : mark EOT token for Granite models

* sampling : fallback to EOS when EOT is not found
This commit is contained in:
Georgi Gerganov
2025-10-10 17:17:31 +03:00
committed by GitHub
parent 68ee98ae18
commit 81086cd6a3
2 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
src/llama-sampling.cpp
View File

@@ -2541,8 +2541,13 @@ static void llama_sampler_infill_apply(struct llama_sampler * smpl, llama_token_
if (n_non_eog == 0) {
cur_p->size = 1;
cur_p->data[0].id = ctx->vocab->token_eot();
if (cur_p->data[0].id == LLAMA_TOKEN_NULL) {
cur_p->data[0].id = ctx->vocab->token_eos();
}
cur_p->data[0].logit = 1.0f;
GGML_ASSERT(cur_p->data[0].id != LLAMA_TOKEN_NULL);
return;
}