Codestral-22B-v0.1/tokenizer.model.metadata at 04ccb687cb40b70cd3128bfb27747b1c59ea7aad - Codestral-22B-v0.1 - Gitea: Git with a cup of tea

LLM-Research/Codestral-22B-v0.1

Files

ai-modelscope 04ccb687cb Align tokenizer with mistral-common (#45 )

- Align tokenizer with mistral-common (53f216c52ce4534a38a71c21861acd514fa8a904)
- Defend the honour of the Hugging Face tokenizer (684c1751c210aa11e0b187c0eac1b7b2bd4d7967)
- Update to tokenizer v3 with correct proper special tokens (106a1b0c338ddbd0e3e42dbeb63634bc85d6f71b)
- Re-add chat template (3256c7e7ea279386e0cdd18553202ed78c4d735b)

Co-authored-by: Matthew Carrigan <Rocketknight1@users.noreply.huggingface.co>

2024-07-31 08:03:57 +08:00

4 lines

125 B

Plaintext

Raw Blame History

	`1c56091ac7f0ab22f0f4af40655eab46e1be34f7`
	`9addc8bdce5988448ae81b729336f43a81262160ae8da760674badab9d4c7d33`
	`1722383969.4252708`