tags, library_name

tags	library_name
causal-lm text-generation	transformers

nikitastheo/phonebabylm-phone-deu-eng-sequential_interleaved

Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).

Training details

• Base config: gpt_base_config.json
• Tokenizer: nikitastheo/phonelm-phone-deu-eng-tokenizer
• Max steps: 30000
• Learning rate: 0.0001
• LR scheduler: linear
• Warmup steps: 3000
• Batch size (per device): 32
• Gradient accumulation steps: 1
• Total train batch size: 32
• Language switch steps: 10000