Serendip-LLM-CPT-SFT-v2/training_scripts

Training Scripts

This folder contains all training scripts used in the SerendipLLM V2 project.

✅ Final Script Used (ACTUAL TRAINING)

train_v2_fast.py ← This is the exact script that trained the final model!

• Training time: 26.5 hours
• Loss: 0.54 → 0.27 (50% improvement)
• Epochs: 3
• Batch size: 8 (effective 32)
• LoRA rank: 64
• Dataset: 309,328 examples

📝 Other Scripts (Development/Testing)

• train_phase1_fixed.py - Initial attempt (slower, 512 tokens)
• continue_training.py - Script for resuming training (not used)

🎯 To Reproduce

Use train_v2_fast.py with these settings:

• GPU: A100 80GB
• Dataset: Chamaka8/Serendip-sft-sinhala (serendipllm_sft_final_train_v2.json)
• Time: ~27 hours
• Cost: ~$37

📊 Training Results

Epoch 1: Loss 0.28
Epoch 2: Loss 0.24
Epoch 3: Loss 0.27
Final average loss: 0.27

🔗 Related Resources

• Model: https://huggingface.co/Chamaka8/Serendip-LLM-CPT-SFT-v2
• Dataset: https://huggingface.co/datasets/Chamaka8/Serendip-sft-sinhala
• Base CPT: https://huggingface.co/Chamaka8/serendib-llm-cpt-llama3-8b