68 lines
2.4 KiB
Markdown
68 lines
2.4 KiB
Markdown
---
|
|
base_model: qwen3
|
|
language:
|
|
- si
|
|
- en
|
|
pipeline_tag: text-generation
|
|
library_name: transformers
|
|
license: apache-2.0
|
|
datasets:
|
|
- your_dataset_name_here
|
|
tags:
|
|
- unsloth
|
|
- qwen
|
|
- qwen3
|
|
- sinhala
|
|
- text-generation
|
|
- custom-finetune
|
|
- causal-lm
|
|
- nlp
|
|
- pytorch
|
|
---
|
|
|
|
# Sinhala Qwen 3 - Fine-Tuned Model (v7500)
|
|
|
|
## Model Description
|
|
This is a fine-tuned version of the **Qwen 3** base model, specifically trained to understand and generate the **Sinhala** language.
|
|
|
|
Originally, the base model had absolutely no understanding of Sinhala. Through custom fine-tuning using the [Unsloth](https://github.com/unslothai/unsloth) library, this model has been taught the foundational elements of the language from scratch. It is now capable of basic Sinhala comprehension and text generation, marking a significant step in low-resource language adaptation for this architecture.
|
|
|
|
## Model Details
|
|
* **Base Model:** Qwen 3 (`Qwen3ForCausalLM`)
|
|
* **Language(s):** Sinhala (si), English (en)
|
|
* **Fine-Tuning Library:** Unsloth (`unsloth_version: 2026.1.4`)
|
|
* **Checkpoint/Version:** v7500 (Trained up to 7500 steps)
|
|
* **Status:** Experimental / Early Stage
|
|
|
|
## Intended Use
|
|
This model is intended for researchers and developers working on Natural Language Processing (NLP) for the Sinhala language. It can be used as a starting point for further fine-tuning, vocabulary expansion, or basic Sinhala text generation tasks.
|
|
|
|
*Note: Since the model was trained from scratch to learn a completely new language, it might still make grammatical errors or occasionally struggle with complex sentence structures. It represents a foundation that is continuously being improved.*
|
|
|
|
## How to Use
|
|
|
|
You can use this model directly with the `transformers` library in Python:
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
|
|
model_id = "sh4lu-z/Sinhala-Qwen3-v7500"
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
|
|
|
|
prompt = "ඔබට කොහොමද?"
|
|
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
|
|
outputs = model.generate(**inputs, max_new_tokens=50)
|
|
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
```
|
|
|
|
## Training Procedure
|
|
|
|
This model was trained using the Unsloth library to significantly speed up the fine-tuning process while maintaining accuracy.
|
|
|
|
#### `Framework:` PyTorch / Transformers
|
|
|
|
#### `Hardware:` Fine-tuned on GPU instances |