Files
legal-clean-model/tokenizer_config.json
ModelHub XC e39ab1276d 初始化项目,由ModelHub XC社区提供模型
Model: Callmeyash11/legal-clean-model
Source: Original Platform
2026-08-11 14:23:17 +08:00

9 lines
222 B
JSON

{
"tokenizer_class": "PreTrainedTokenizerFast",
"model_max_length": 2048,
"padding_side": "left",
"bos_token": "<|begin_of_text|>",
"eos_token": "<|eot_id|>",
"pad_token": "<|eot_id|>",
"unk_token": "<unk>"
}