brahmastra-0.2/tokenizer_config.json at 6c694cc4cc297c2070489dcb00ee46347b9ad5ba - brahmastra-0.2 - Gitea: Git with a cup of tea

Krishnapadala55/brahmastra-0.2

Files

ModelHub XC 6c694cc4cc 初始化项目，由ModelHub XC社区提供模型

Model: Krishnapadala55/brahmastra-0.2
Source: Original Platform

2026-06-14 22:06:17 +08:00

11 lines

277 B

JSON

Raw Blame History

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

 {
   "backend": "tokenizers",
   "bos_token": "<｜begin▁of▁sentence｜>",
   "eos_token": "<|im_end|>",
   "is_local": true,
   "model_max_length": 1000000000000000019884624838656,
   "pad_token": "<|PAD_TOKEN|>",
   "tokenizer_class": "TokenizersBackend",
   "unk_token": null
 }