language, pipeline_tag, base_model, license

language	pipeline_tag	base_model	license
et en	text-generation	meta-llama/Llama-2-7b-hf	llama2

LLammas-base 🐑

Llama-2-7B with continued pre-training of 5B tokens of CulturaX (75% Estonian, 25% English documents).

This model is also instruction-tuned resulting in Llammas.

More details in our paper.

Citation

@misc{kuulmets2024teaching,
      title={Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer}, 
      author={Hele-Andra Kuulmets and Taido Purason and Agnes Luhtaru and Mark Fishel},
      year={2024},
      eprint={2404.04042},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}