LLM-Research/Codestral-22B-v0.1
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
04ccb687cb40b70cd3128bfb27747b1c59ea7aad
Codestral-22B-v0.1/.huggingface/download/tokenizer.json.metadata

4 lines
101 B
Plaintext
Raw Normal View History

Align tokenizer with mistral-common (#45) - Align tokenizer with mistral-common (53f216c52ce4534a38a71c21861acd514fa8a904) - Defend the honour of the Hugging Face tokenizer (684c1751c210aa11e0b187c0eac1b7b2bd4d7967) - Update to tokenizer v3 with correct proper special tokens (106a1b0c338ddbd0e3e42dbeb63634bc85d6f71b) - Re-add chat template (3256c7e7ea279386e0cdd18553202ed78c4d735b) Co-authored-by: Matthew Carrigan <Rocketknight1@users.noreply.huggingface.co>
2024-07-31 08:03:57 +08:00
1c56091ac7f0ab22f0f4af40655eab46e1be34f7
387cfa3eb95ef70c2061ea94aaf58d7fdc8f483d
1722383969.2597454
Reference in New Issue Copy Permalink