Files
tournament-llama-test-001-e…/tokenizer_config.json
ModelHub XC 92d95345a5 初始化项目,由ModelHub XC社区提供模型
Model: gradients-io-tournaments/tournament-llama-test-001-eeb0087a-e949-4294-966b-658ed1f61fee-5CMPlate
Source: Original Platform
2026-09-19 17:54:15 +08:00

21 lines
475 B
JSON

{
"add_prefix_space": false,
"backend": "tokenizers",
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|endoftext|>",
"errors": "replace",
"extra_special_tokens": [
"<|im_start|>",
"<|im_end|>"
],
"is_local": true,
"local_files_only": false,
"model_max_length": 32768,
"pad_token": "<|PAD_TOKEN|>",
"padding_side": "left",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}