初始化项目,由ModelHub XC社区提供模型
Model: Chamaka8/Serendip-LLM-CPT-SFT-v2 Source: Original Platform
This commit is contained in:
52
training_scripts/TRAINING_LOG.md
Normal file
52
training_scripts/TRAINING_LOG.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# Training Log Summary
|
||||
|
||||
## Final Training Run (train_v2_fast.py)
|
||||
|
||||
**Start:** February 18, 2026, ~17:30
|
||||
**End:** February 19, 2026, ~20:00
|
||||
**Duration:** 26.5 hours
|
||||
|
||||
### Loss Progression
|
||||
|
||||
| Epoch | Loss |
|
||||
|-------|------|
|
||||
| 0.95 | 0.28 |
|
||||
| 1.90 | 0.24 |
|
||||
| 3.00 | 0.27 |
|
||||
|
||||
**Final training loss:** 0.27
|
||||
|
||||
### Configuration Used
|
||||
```python
|
||||
num_train_epochs = 3
|
||||
per_device_train_batch_size = 8
|
||||
gradient_accumulation_steps = 4 # Effective batch = 32
|
||||
learning_rate = 2e-5
|
||||
max_length = 384 # tokens
|
||||
warmup_steps = 200
|
||||
weight_decay = 0.01
|
||||
|
||||
# LoRA Config
|
||||
lora_r = 64
|
||||
lora_alpha = 128
|
||||
lora_target_modules = [
|
||||
"q_proj", "k_proj", "v_proj",
|
||||
"o_proj", "gate_proj", "up_proj"
|
||||
]
|
||||
lora_dropout = 0.05
|
||||
```
|
||||
|
||||
### Hardware
|
||||
|
||||
- GPU: NVIDIA A100 SXM 80GB
|
||||
- Training framework: Transformers + PEFT
|
||||
- Mixed precision: FP16
|
||||
|
||||
### Dataset
|
||||
|
||||
- Source: Chamaka8/Serendip-sft-sinhala
|
||||
- File: serendipllm_sft_final_train_v2.json
|
||||
- Examples: 309,328
|
||||
- News classification: 45,080 examples
|
||||
- General Sinhala: 205,403 examples
|
||||
- QA pairs: 29,390 examples
|
||||
Reference in New Issue
Block a user