license, language, pipeline_tag, library_name, datasets
| license | language | pipeline_tag | library_name | datasets | ||
|---|---|---|---|---|---|---|
| mit |
|
text-generation | transformers |
|
Vintage-LLM
This is a 340M params Llama3-based model, trained on text pre-1900.
This is a base model. It has very limited chat capabilities.
Dataset used: https://huggingface.co/datasets/croqaz/vintage-v1
The data was de-duplicated and only the decent quality texts were used.
There are a few checkpoints in the repo:
- 10k steps on short texts: commit/ff3b94efdaccfa312754d3a4328b7c7a05155d36
- previous model + short ft on vintage-FT-v1: commit/3946781d51e76c7cf0874bd595003bce1755cbd2
- previous model + 10k steps on long texts: commit/c9b48cc3aaa4b3dc0ea3b846f45e98a05e94c8cb