Codestral-22B-v0.1/tokenizer.model

version https://git-lfs.github.com/spec/v1
oid sha256:9addc8bdce5988448ae81b729336f43a81262160ae8da760674badab9d4c7d33
size 587591
Add instructions on how to run the model with transformers (#31) - Update README.md (f2b0fc04dc4a811841bab00bcc5986d4a83004b2) Co-authored-by: Younes Belkada <ybelkada@users.noreply.huggingface.co> 2024-06-04 22:20:50 +08:00			`version https://git-lfs.github.com/spec/v1`
Align tokenizer with mistral-common (#45) - Align tokenizer with mistral-common (53f216c52ce4534a38a71c21861acd514fa8a904) - Defend the honour of the Hugging Face tokenizer (684c1751c210aa11e0b187c0eac1b7b2bd4d7967) - Update to tokenizer v3 with correct proper special tokens (106a1b0c338ddbd0e3e42dbeb63634bc85d6f71b) - Re-add chat template (3256c7e7ea279386e0cdd18553202ed78c4d735b) Co-authored-by: Matthew Carrigan <Rocketknight1@users.noreply.huggingface.co> 2024-07-31 08:03:57 +08:00			`oid sha256:9addc8bdce5988448ae81b729336f43a81262160ae8da760674badab9d4c7d33`
			`size 587591`