初始化项目,由ModelHub XC社区提供模型
Model: EdonFetaji/MK-Llama-3.2-1B Source: Original Platform
This commit is contained in:
31
README.md
Normal file
31
README.md
Normal file
@@ -0,0 +1,31 @@
|
||||
---
|
||||
license: llama3.2
|
||||
datasets:
|
||||
- LVSTCK/macedonian-corpus-cleaned-dedup
|
||||
language:
|
||||
- mk
|
||||
base_model:
|
||||
- meta-llama/Llama-3.2-1B
|
||||
tags:
|
||||
- mk
|
||||
- continual-pretraining
|
||||
- lora
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
# MK Llama-3.2-1B (Continued Pretraining)
|
||||
## Summary
|
||||
Continued pretraining for Macedonian language on **lvstck/macedonian-corpus-cleaned-dedup**.
|
||||
Trained using LoRA adapters on a **single A100**.
|
||||
## Training
|
||||
- Stage 1: **0.4 epochs** on Google Colab A100
|
||||
- Stage 2: **0.9 epochs** on A100 - FINKI GPU cluster
|
||||
- Total: **1.3 epochs**
|
||||
## Training Metrics
|
||||
TensorBoard logs are available in the **Training metrics** tab of this model repository. **Logs only available for STAGE 2**
|
||||
The logs include training loss, learning rate, gradient norm, and evaluation metrics .
|
||||
|
||||
- View the TensorBoard dashboard here:
|
||||
[Open TensorBoard training metrics](https://huggingface.co/EdonFetaji/MK-Llama-3.2-1B/tensorboard)
|
||||
## Notes
|
||||
This is a continued pretraining checkpoint focused on Macedonian.
|
||||
Reference in New Issue
Block a user