tags, library_name
| tags | library_name | ||
|---|---|---|---|
|
transformers |
nikitastheo/goldfish-ita-ell-sequential_interleaved
Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).
Training details
- Base config:
gpt_base_config.json - Tokenizer:
nikitastheo/goldfish-ita-ell-tokenizer - Max steps: 22050
- Learning rate: 0.0001
- LR scheduler: linear
- Warmup steps: 2205
- Batch size (per device): 32
- Gradient accumulation steps: 1
- Total train batch size: 32
- Language switch epoch: 10