nikitastheo/babylm-lem-fra-ell-sequential_interleaved
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
babylm-lem-fra-ell-sequenti…/README.md
ModelHub XC e402834cb4 初始化项目，由ModelHub XC社区提供模型 
Model: nikitastheo/babylm-lem-fra-ell-sequential_interleaved
Source: Original Platform
2026-08-05 15:28:37 +08:00

588 B
Raw Permalink Blame History

tags, library_name
tags library_name
causal-lm
text-generation
transformers

nikitastheo/babylm-lem-fra-ell-sequential_interleaved

Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).

Training details

  • Base config: gpt_base_config.json
  • Tokenizer: nikitastheo/babylm-lem-fra-ell-tokenizer
  • Max steps: 26090
  • Learning rate: 0.0001
  • LR scheduler: linear
  • Warmup steps: 2609
  • Batch size (per device): 32
  • Gradient accumulation steps: 1
  • Total train batch size: 32
  • Language switch epoch: 10