46 lines
1.0 KiB
Markdown
46 lines
1.0 KiB
Markdown
---
|
|
language:
|
|
- en
|
|
license: mit
|
|
tags:
|
|
- gpt2
|
|
- text-generation
|
|
- historical
|
|
- australian
|
|
- pytorch
|
|
---
|
|
|
|
# Chronicle LLM v0 🇦🇺
|
|
|
|
A GPT-style language model trained from scratch on Australian texts from 1850-1950.
|
|
No fine-tuning. No modern weights. Built entirely from historical Australian writing.
|
|
|
|
## Model Details
|
|
|
|
- **Architecture:** GPT-2 decoder-only transformer
|
|
- **Parameters:** 30M
|
|
- **Training data:** 141 verified Australian texts, 55MB cleaned, ~14M tokens
|
|
- **Training steps:** 20,000
|
|
- **Final train loss:** 2.81
|
|
- **Final val loss:** 4.68
|
|
|
|
## Files
|
|
|
|
- `model.safetensors` - model weights (HuggingFace format)
|
|
- `chronicle_v0.gguf` - GGUF format for LM Studio and llama.cpp
|
|
|
|
## Usage
|
|
|
|
Load in LM Studio using the GGUF file, or via API:
|
|
|
|
```python
|
|
from transformers import GPT2LMHeadModel, GPT2Tokenizer
|
|
|
|
model = GPT2LMHeadModel.from_pretrained("Gnayo/chronicle-llm-v0")
|
|
tokenizer = GPT2Tokenizer.from_pretrained("Gnayo/chronicle-llm-v0")
|
|
```
|
|
|
|
## GitHub
|
|
|
|
Full training code and documentation:
|
|
https://github.com/ravipatib/ChronicleLLM |