初始化项目,由ModelHub XC社区提供模型

Model: SAWithanage/SinLlama-Llama-3-8B-Merged
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-11 00:50:17 +08:00
commit e1efdabf06
16 changed files with 496 additions and 0 deletions

77
README.md Normal file
View File

@@ -0,0 +1,77 @@
---
language:
- si
- en
base_model: meta-llama/Meta-Llama-3-8B
tags:
- llama-3
- sinhala
- nlp
- text-generation
- merged
license: llama3
---
# SinLlama Llama-3-8B Merged
This repository contains a continuous pre-trained (CPT) base model for the Sinhala language. It was created by mathematically merging the official `meta-llama/Meta-Llama-3-8B` base model with the `polyglots/SinLlama_v01` LoRA adapter.
**This model has been fully merged into standalone FP16 weights**, meaning it can be loaded directly in libraries like `transformers` or `vLLM` without needing to download or manage separate Peft adapters.
## 🏆 Acknowledgments, Credits & Disclaimer
**My contribution to this repository is strictly limited to merging the weights to provide a convenient standalone model.**
All credit for the foundational machine learning research, tokenizer vocabulary expansion, dataset curation, and continuous pre-training (CPT) belongs entirely to the **[Polyglots](https://huggingface.co/polyglots)** team and the authors of the SinLlama paper.
* **Researchers/Authors:** H. W. K. Aravinda, Rashad Sirajudeen, Samith Karunathilake, Nisansa de Silva, Surangika Ranathunga, Rishemjit Kaur
* **Original Adapter:** [polyglots/SinLlama_v01](https://huggingface.co/polyglots/SinLlama_v01)
* **Original Tokenizer:** [polyglots/Extended-Sinhala-LLaMA](https://huggingface.co/polyglots/Extended-Sinhala-LLaMA)
* **Paper:** *SinLlama: A Large Language Model for Sinhala*
If you use this model in your research or applications, please ensure you cite their original work:
```bibtex
@article{aravinda2025sinllama,
title={SinLlama-A Large Language Model for Sinhala},
author={Aravinda, H W K and Sirajudeen, Rashad and Karunathilake, Samith and de Silva, Nisansa and Ranathunga, Surangika and Kaur, Rishemjit},
journal={arXiv preprint arXiv:2508.09115},
year={2025}
}
```
## ⚙️ Model Details
* **Base Model:** [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B)
* **Language:** Sinhala (si), English (en)
* **Architecture:** Llama 3 (8 Billion Parameters)
* **Format:** Safetensors (Unquantized FP16)
## ⚠️ Important Limitations (Base Model vs. Chat Model)
**This is a Base Model, not an Instruction-Tuned (Chat) Model.**
Because it has not undergone Supervised Fine-Tuning (SFT), it is optimized for text completion, not interactive conversation.
If you prompt it with a question, it may attempt to complete the document by writing more questions (acting like an FAQ page). To use it effectively for Q&A, you must format your prompts strictly and use stopping criteria (like regex or EOS tokens) to prevent looping and run-on generations.
### Example Prompt Format:
```text
Question: කෘතිම බුද්ධිය (AI) යනු කුමක්දැයි සරලව පැහැදිලි කරන්න.
Answer:
```
## 💻 How to Load in 4-bit (Google Colab / T4 GPU)
```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.float16
)
tokenizer = AutoTokenizer.from_pretrained("SAWithanage/SinLlama-Llama-3-8B-Merged")
model = AutoModelForCausalLM.from_pretrained(
"SAWithanage/SinLlama-Llama-3-8B-Merged",
quantization_config=bnb_config,
device_map="auto"
)
```