94 lines
2.4 KiB
Markdown
94 lines
2.4 KiB
Markdown
|
---
|
||
|
|
language:
|
||
|
|
- pcm
|
||
|
|
- en
|
||
|
|
license: apache-2.0
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: transformers
|
||
|
|
base_model: Qwen/Qwen3-4B
|
||
|
|
tags:
|
||
|
|
- Nigerian-Pidgin
|
||
|
|
- Naija
|
||
|
|
- Pidgin-English
|
||
|
|
- Africa
|
||
|
|
- unsloth
|
||
|
|
- fine-tuned
|
||
|
|
- qwen3
|
||
|
|
datasets:
|
||
|
|
- ahmadabdulnasir/naijaPidgin
|
||
|
|
---
|
||
|
|
|
||
|
|
# NaijaPidgin-Qwen3-4B 🇳🇬
|
||
|
|
|
||
|
|
A fine-tuned Qwen3-4B model that understands and speaks **Nigerian Pidgin English (Naija)** fluently.
|
||
|
|
|
||
|
|
## Model Description
|
||
|
|
|
||
|
|
This model was fine-tuned from `Qwen/Qwen3-4B` using [Unsloth](https://github.com/unslothai/unsloth)
|
||
|
|
with QLoRA on the [naijaPidgin dataset](https://huggingface.co/datasets/ahmadabdulnasir/naijaPidgin).
|
||
|
|
|
||
|
|
**Capabilities:**
|
||
|
|
- Conversational Pidgin English
|
||
|
|
- Pidgin ↔ English translation
|
||
|
|
- Nigerian culture, proverbs, and everyday advice
|
||
|
|
- Code-switching between Pidgin and English
|
||
|
|
- Thinking mode (Qwen3 feature) for complex questions
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model = AutoModelForCausalLM.from_pretrained("ahmadabdulnasir/NaijaPidgin-Qwen3-4B")
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("ahmadabdulnasir/NaijaPidgin-Qwen3-4B")
|
||
|
|
|
||
|
|
messages = [
|
||
|
|
{"role": "system", "content": "You are a helpful assistant that speaks Nigerian Pidgin English. /no_think"},
|
||
|
|
{"role": "user", "content": "How you dey?"},
|
||
|
|
]
|
||
|
|
|
||
|
|
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
|
||
|
|
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, top_p=0.8, top_k=20)
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
### With Thinking Mode
|
||
|
|
|
||
|
|
Remove `/no_think` from the system prompt to enable Qwen3's reasoning mode:
|
||
|
|
|
||
|
|
```python
|
||
|
|
messages = [
|
||
|
|
{"role": "system", "content": "You are a helpful assistant that speaks Nigerian Pidgin English."},
|
||
|
|
{"role": "user", "content": "If I get 50,000 naira, wetin be di best investment?"},
|
||
|
|
]
|
||
|
|
```
|
||
|
|
|
||
|
|
### With Ollama
|
||
|
|
|
||
|
|
Download the GGUF version and run locally:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
ollama run ahmadabdulnasir/NaijaPidgin-Qwen3-4B
|
||
|
|
```
|
||
|
|
|
||
|
|
## Training Details
|
||
|
|
|
||
|
|
- **Base model:** Qwen/Qwen3-4B
|
||
|
|
- **Method:** QLoRA (4-bit) with Unsloth
|
||
|
|
- **LoRA rank:** 32
|
||
|
|
- **Learning rate:** 2e-4
|
||
|
|
- **Epochs:** 3
|
||
|
|
- **Hardware:** Google Colab G4 GPU (96GB VRAM)
|
||
|
|
- **Dataset:** ahmadabdulnasir/naijaPidgin
|
||
|
|
|
||
|
|
## Citation
|
||
|
|
|
||
|
|
```bibtex
|
||
|
|
@misc{naijapidgin_qwen3_2025,
|
||
|
|
title={NaijaPidgin-Qwen3-4B},
|
||
|
|
author={Ahmad Abdulnasir Shuaib},
|
||
|
|
year={2025},
|
||
|
|
url={https://huggingface.co/ahmadabdulnasir/NaijaPidgin-Qwen3-4B}
|
||
|
|
}
|
||
|
|
```
|