169 lines
3.2 KiB
Markdown
169 lines
3.2 KiB
Markdown
|
---
|
||
|
|
language:
|
||
|
|
- ber
|
||
|
|
- tmz
|
||
|
|
- ar
|
||
|
|
- en
|
||
|
|
license: apache-2.0
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- tamazight
|
||
|
|
- amazigh
|
||
|
|
- berber
|
||
|
|
- llm
|
||
|
|
- nlp
|
||
|
|
- qwen
|
||
|
|
- chatbot
|
||
|
|
- low-resource-language
|
||
|
|
- morocco
|
||
|
|
- africa
|
||
|
|
- text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# AWAL GPT v0.2 7B
|
||
|
|
|
||
|
|
AWAL GPT v0.2 is an open-source Large Language Model fine-tuned for the Amazigh (Tamazight) language, with a primary focus on **Latin-script Tamazight used in Morocco**.
|
||
|
|
|
||
|
|
To the best of the author's knowledge, this is **the first open-source Amazigh (Tamazight Latin script) LLM developed in Morocco**.
|
||
|
|
|
||
|
|
The project aims to contribute to the preservation, digitization, and advancement of Amazigh through Artificial Intelligence and Natural Language Processing.
|
||
|
|
|
||
|
|
## Project Goals
|
||
|
|
|
||
|
|
AWAL GPT is part of a larger initiative to build a complete AI ecosystem for Tamazight, including:
|
||
|
|
|
||
|
|
- Large Language Models (LLMs)
|
||
|
|
- Intelligent Chatbots
|
||
|
|
- Speech-to-Text (ASR)
|
||
|
|
- Text-to-Speech (TTS)
|
||
|
|
- Machine Translation
|
||
|
|
- Grammar Correction
|
||
|
|
- OCR
|
||
|
|
- Retrieval-Augmented Generation (RAG)
|
||
|
|
- Voice Assistants
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Base Model
|
||
|
|
|
||
|
|
This model is fine-tuned from:
|
||
|
|
|
||
|
|
- Qwen2.5-7B
|
||
|
|
|
||
|
|
using instruction tuning on Amazigh datasets.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Supported Language
|
||
|
|
|
||
|
|
Current focus:
|
||
|
|
|
||
|
|
- Moroccan Tamazight (Latin script)
|
||
|
|
|
||
|
|
Future versions aim to support:
|
||
|
|
|
||
|
|
- Tifinagh
|
||
|
|
- Arabic script
|
||
|
|
- Multiple Amazigh dialects (Tashelhit, Central Atlas Tamazight, Tarifit)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Example
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
|
|
||
|
|
model_name = "mohamedazelmad/awal-gpt-v0.2-7b"
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(model_name)
|
||
|
|
|
||
|
|
prompt = "Mani ismk?"
|
||
|
|
|
||
|
|
inputs = tokenizer(prompt, return_tensors="pt")
|
||
|
|
|
||
|
|
outputs = model.generate(
|
||
|
|
**inputs,
|
||
|
|
max_new_tokens=128
|
||
|
|
)
|
||
|
|
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Intended Uses
|
||
|
|
|
||
|
|
- Research
|
||
|
|
- Education
|
||
|
|
- Amazigh NLP
|
||
|
|
- Chatbots
|
||
|
|
- Language preservation
|
||
|
|
- AI experimentation
|
||
|
|
- Academic projects
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Limitations
|
||
|
|
|
||
|
|
- The model is still under active development.
|
||
|
|
- Performance may vary across different Amazigh dialects.
|
||
|
|
- Responses may contain factual inaccuracies or hallucinations.
|
||
|
|
- Not intended for high-risk applications without human verification.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Roadmap (AWAL GPT V2)
|
||
|
|
|
||
|
|
The next generation will include:
|
||
|
|
|
||
|
|
- Improved reasoning
|
||
|
|
- Larger training corpus
|
||
|
|
- Better instruction following
|
||
|
|
- Speech capabilities (ASR/TTS)
|
||
|
|
- Translation system
|
||
|
|
- OCR
|
||
|
|
- RAG integration
|
||
|
|
- Voice assistant
|
||
|
|
- Public web interface
|
||
|
|
- WhatsApp integration
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Citation
|
||
|
|
|
||
|
|
If you use this model in your research, please cite:
|
||
|
|
|
||
|
|
```bibtex
|
||
|
|
@misc{awalgpt2026,
|
||
|
|
title={AWAL GPT: Large Language Models for Amazigh (Tamazight)},
|
||
|
|
author={Mohamed Azelmad},
|
||
|
|
year={2026},
|
||
|
|
publisher={Hugging Face},
|
||
|
|
url={https://huggingface.co/mohamedazelmad/awal-gpt-v0.2-7b}
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Contact
|
||
|
|
|
||
|
|
**Author:** Mohamed Azelmad
|
||
|
|
|
||
|
|
For collaborations, research partnerships, or contributions related to Amazigh AI and NLP, feel free to connect via LinkedIn or Hugging Face.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Acknowledgements
|
||
|
|
|
||
|
|
This project builds upon open-source technologies from the AI community, including:
|
||
|
|
|
||
|
|
- Qwen
|
||
|
|
- Hugging Face Transformers
|
||
|
|
- PyTorch
|
||
|
|
- TRL
|
||
|
|
- PEFT
|
||
|
|
- Unsloth
|
||
|
|
|
||
|
|
Special thanks to everyone working on low-resource languages and open AI research.
|