nikitastheo/babylm-lem-spa-ell-sequential_interleaved
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
babylm-lem-spa-ell-sequenti…/README.md
ModelHub XC 7b374301db 初始化项目，由ModelHub XC社区提供模型 
Model: nikitastheo/babylm-lem-spa-ell-sequential_interleaved
Source: Original Platform
2026-08-05 15:29:21 +08:00

588 B
Raw Permalink Blame History

tags, library_name
tags library_name
causal-lm
text-generation
transformers

nikitastheo/babylm-lem-spa-ell-sequential_interleaved

Trained with train_clm.py, a Hugging Face Accelerate causal-LM training script (no Trainer).

Training details

  • Base config: gpt_base_config.json
  • Tokenizer: nikitastheo/babylm-lem-spa-ell-tokenizer
  • Max steps: 25460
  • Learning rate: 0.0001
  • LR scheduler: linear
  • Warmup steps: 2546
  • Batch size (per device): 32
  • Gradient accumulation steps: 1
  • Total train batch size: 32
  • Language switch epoch: 10