MK-Llama-3.2-1B/README.md

license, datasets, language, base_model, tags, library_name

license	datasets	language	base_model	tags	library_name
llama3.2	LVSTCK/macedonian-corpus-cleaned-dedup	mk	meta-llama/Llama-3.2-1B	mk continual-pretraining lora	transformers

MK Llama-3.2-1B (Continued Pretraining)

Summary

Continued pretraining for Macedonian language on lvstck/macedonian-corpus-cleaned-dedup.
Trained using LoRA adapters on a single A100.

Training

• Stage 1: 0.4 epochs on Google Colab A100
• Stage 2: 0.9 epochs on A100 - FINKI GPU cluster
• Total: 1.3 epochs

Training Metrics

TensorBoard logs are available in the Training metrics tab of this model repository. Logs only available for STAGE 2 The logs include training loss, learning rate, gradient norm, and evaluation metrics .

• View the TensorBoard dashboard here: Open TensorBoard training metrics

Notes

This is a continued pretraining checkpoint focused on Macedonian.