tags, library_name
| tags | library_name | ||
|---|---|---|---|
|
transformers |
nikitastheo/phonebabylm-phone-deu-eng-interleaved
Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).
Training details
- Base config:
gpt_base_config.json - Tokenizer:
nikitastheo/phonelm-phone-deu-eng-tokenizer - Max steps: 40000
- Learning rate: 0.0005
- LR scheduler: linear
- Warmup steps: 4000
- Batch size (per device): 32
- Gradient accumulation steps: 1
- Total train batch size: 32