---
license: llama3.2
datasets:
- LVSTCK/macedonian-corpus-cleaned-dedup
language:
- mk
base_model:
- meta-llama/Llama-3.2-1B
tags:
- mk
- continual-pretraining
- lora
library_name: transformers
---

# MK Llama-3.2-1B (Continued Pretraining)
## Summary
Continued pretraining for Macedonian language on **lvstck/macedonian-corpus-cleaned-dedup**.  
Trained using LoRA adapters on a **single A100**.
## Training
- Stage 1: **0.4 epochs** on Google Colab A100  
- Stage 2: **0.9 epochs** on A100 - FINKI GPU cluster  
- Total: **1.3 epochs**
## Training Metrics
TensorBoard logs are available in the **Training metrics** tab of this model repository. **Logs only available for STAGE 2**
The logs include training loss, learning rate, gradient norm, and evaluation metrics .

- View the TensorBoard dashboard here:
 [Open TensorBoard training metrics](https://huggingface.co/EdonFetaji/MK-Llama-3.2-1B/tensorboard)
## Notes
This is a continued pretraining checkpoint focused on Macedonian.