初始化项目,由ModelHub XC社区提供模型

Model: EdonFetaji/MK-Llama-3.2-1B
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-15 08:15:24 +08:00
commit c94b3174ab
9 changed files with 2199 additions and 0 deletions

31
README.md Normal file
View File

@@ -0,0 +1,31 @@
---
license: llama3.2
datasets:
- LVSTCK/macedonian-corpus-cleaned-dedup
language:
- mk
base_model:
- meta-llama/Llama-3.2-1B
tags:
- mk
- continual-pretraining
- lora
library_name: transformers
---
# MK Llama-3.2-1B (Continued Pretraining)
## Summary
Continued pretraining for Macedonian language on **lvstck/macedonian-corpus-cleaned-dedup**.
Trained using LoRA adapters on a **single A100**.
## Training
- Stage 1: **0.4 epochs** on Google Colab A100
- Stage 2: **0.9 epochs** on A100 - FINKI GPU cluster
- Total: **1.3 epochs**
## Training Metrics
TensorBoard logs are available in the **Training metrics** tab of this model repository. **Logs only available for STAGE 2**
The logs include training loss, learning rate, gradient norm, and evaluation metrics .
- View the TensorBoard dashboard here:
[Open TensorBoard training metrics](https://huggingface.co/EdonFetaji/MK-Llama-3.2-1B/tensorboard)
## Notes
This is a continued pretraining checkpoint focused on Macedonian.