---
tags:
- causal-lm
- text-generation
library_name: transformers
---

# nikitastheo/phonebabylm-phone-deu-eng-sequential_interleaved

Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no `Trainer`).

## Training details

- **Base config**: `gpt_base_config.json`
- **Tokenizer**: `nikitastheo/phonelm-phone-deu-eng-tokenizer`
- **Max steps**: 30000
- **Learning rate**: 0.0001
- **LR scheduler**: linear
- **Warmup steps**: 3000
- **Batch size (per device)**: 32
- **Gradient accumulation steps**: 1
- **Total train batch size**: 32
- **Language switch steps**: 10000