初始化项目,由ModelHub XC社区提供模型
Model: Chamaka8/Serendip-LLM-CPT-SFT-v2 Source: Original Platform
This commit is contained in:
41
training_scripts/README.md
Normal file
41
training_scripts/README.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Training Scripts
|
||||
|
||||
This folder contains all training scripts used in the SerendipLLM V2 project.
|
||||
|
||||
## ✅ Final Script Used (ACTUAL TRAINING)
|
||||
|
||||
**`train_v2_fast.py`** ← This is the exact script that trained the final model!
|
||||
|
||||
- Training time: 26.5 hours
|
||||
- Loss: 0.54 → 0.27 (50% improvement)
|
||||
- Epochs: 3
|
||||
- Batch size: 8 (effective 32)
|
||||
- LoRA rank: 64
|
||||
- Dataset: 309,328 examples
|
||||
|
||||
## 📝 Other Scripts (Development/Testing)
|
||||
|
||||
- `train_phase1_fixed.py` - Initial attempt (slower, 512 tokens)
|
||||
- `continue_training.py` - Script for resuming training (not used)
|
||||
|
||||
## 🎯 To Reproduce
|
||||
|
||||
Use `train_v2_fast.py` with these settings:
|
||||
- GPU: A100 80GB
|
||||
- Dataset: Chamaka8/Serendip-sft-sinhala (serendipllm_sft_final_train_v2.json)
|
||||
- Time: ~27 hours
|
||||
- Cost: ~$37
|
||||
|
||||
## 📊 Training Results
|
||||
```
|
||||
Epoch 1: Loss 0.28
|
||||
Epoch 2: Loss 0.24
|
||||
Epoch 3: Loss 0.27
|
||||
Final average loss: 0.27
|
||||
```
|
||||
|
||||
## 🔗 Related Resources
|
||||
|
||||
- Model: https://huggingface.co/Chamaka8/Serendip-LLM-CPT-SFT-v2
|
||||
- Dataset: https://huggingface.co/datasets/Chamaka8/Serendip-sft-sinhala
|
||||
- Base CPT: https://huggingface.co/Chamaka8/serendib-llm-cpt-llama3-8b
|
||||
Reference in New Issue
Block a user