license, datasets, language, base_model, tags, library_name
| license | datasets | language | base_model | tags | library_name | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| llama3.2 |
|
|
|
|
transformers |
MK Llama-3.2-1B (Continued Pretraining)
Summary
Continued pretraining for Macedonian language on lvstck/macedonian-corpus-cleaned-dedup.
Trained using LoRA adapters on a single A100.
Training
- Stage 1: 0.4 epochs on Google Colab A100
- Stage 2: 0.9 epochs on A100 - FINKI GPU cluster
- Total: 1.3 epochs
Training Metrics
TensorBoard logs are available in the Training metrics tab of this model repository. Logs only available for STAGE 2 The logs include training loss, learning rate, gradient norm, and evaluation metrics .
- View the TensorBoard dashboard here: Open TensorBoard training metrics
Notes
This is a continued pretraining checkpoint focused on Macedonian.