Files
kogpt2/tokenizer_config.json

1 line
109 B
JSON
Raw Permalink Normal View History

{"do_lower_case": false, "unk_token": "<unk>", "pad_token": "<pad>", "bos_token": "<s>", "eos_token": "</s>"}