---
tags:
- causal-lm
- text-generation
library_name: transformers
---

# nikitastheo/babylm-afr-ell-sequential_interleaved

Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no `Trainer`).

## Training details

- **Base config**: `gpt_base_config.json`
- **Tokenizer**: `nikitastheo/babylm-afr-ell-tokenizer`
- **Max steps**: 24270
- **Learning rate**: 0.0001
- **LR scheduler**: linear
- **Warmup steps**: 2427
- **Batch size (per device)**: 32
- **Gradient accumulation steps**: 1
- **Total train batch size**: 32
- **Language switch epoch**: 10