初始化项目，由ModelHub XC社区提供模型
Model: RightNowAI/RightNow-Arabic-0.5B-Turbo Source: Original Platform
This commit is contained in:
ModelHub XC
235
README.md
Normal file
235
README.md
Normal file
@@ -0,0 +1,235 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- ar
|
||||
- en
|
||||
base_model: Qwen/Qwen2.5-0.5B
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- arabic
|
||||
- edge
|
||||
- small-language-model
|
||||
- sft
|
||||
- dpo
|
||||
- gguf
|
||||
- qwen2
|
||||
library_name: transformers
|
||||
model-index:
|
||||
- name: RightNow-Arabic-0.5B-Turbo
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
name: COPA-ar
|
||||
dataset:
|
||||
name: copa_ar
|
||||
type: copa_ar
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: 58.4
|
||||
name: Accuracy (norm)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Arabic HellaSwag
|
||||
dataset:
|
||||
name: arabic_mt_hellaswag
|
||||
type: arabic_mt_hellaswag
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: 26.0
|
||||
name: Accuracy (norm)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: ArabicMMLU
|
||||
dataset:
|
||||
name: arabic_leaderboard_arabic_mmlu
|
||||
type: arabic_leaderboard_arabic_mmlu
|
||||
metrics:
|
||||
- type: acc
|
||||
value: 23.2
|
||||
name: Accuracy
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
# RightNow-Arabic-0.5B-Turbo
|
||||
|
||||
### The smallest open Arabic-specialized decoder LLM
|
||||
|
||||
**518M parameters | 398 MB on disk (q4_k_m) | 635 tok/s on H100**
|
||||
|
||||
*Built by [RightNow AI](https://rightnowai.co)*
|
||||
|
||||
<img src="benchmark_pareto.png" width="700">
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
## What is this?
|
||||
|
||||
RightNow-Arabic-0.5B-Turbo is a **518M-parameter Arabic-specialized language model** built on top of Qwen2.5-0.5B via vocabulary injection, continued pretraining, supervised fine-tuning, and direct preference optimization. It is the **smallest open Arabic-specialized decoder LLM** with publicly available weights.
|
||||
|
||||
The model targets **edge deployment**: phones, laptops, embedded devices, and browsers. Quantized to q4_k_m it fits in 398 MB and generates 635 tokens/s at batch size 1.
|
||||
|
||||
## Key Features
|
||||
|
||||
- **27,032 new Arabic tokens** added via mean-subtoken initialization, cutting Arabic tokenizer fertility by 17.3% (2.18 to 1.80 tokens/word)
|
||||
- **504M Arabic pretraining tokens** (Arabic Wikipedia) on 8xH100 SXM5 with FSDP + FlashAttention varlen + Liger fused kernels
|
||||
- **129,116 Arabic instruction pairs** for SFT with response-only loss masking
|
||||
- **6,750 Arabic preference pairs** for DPO
|
||||
- **Weight soup merging** (DPO 50%, SFT 25%, Pretrain 25%) for optimal accuracy
|
||||
- **4 GGUF quantizations** for instant edge deployment
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Evaluated with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) v0.4.11, `apply_chat_template=True`, `limit=200`, `acc_norm` preferred.
|
||||
|
||||
### Head-to-head comparison
|
||||
|
||||
| Model | Params | COPA-ar | HellaSwag-ar | ArabicMMLU | Mean |
|
||||
|-------|--------|---------|-------------|------------|------|
|
||||
| **RightNow-Arabic-0.5B-Turbo (ours)** | **518M** | **58.4%** | **26.0%** | 23.2% | **35.9%** |
|
||||
| Qwen2.5-0.5B-Instruct | 494M | 53.9% | 22.5% | **26.0%** | 34.1% |
|
||||
| Falcon-H1-0.5B-Instruct | 524M | 44.9% | 23.0% | 24.2% | 30.7% |
|
||||
| Falcon-H1-1.5B-Instruct | 1.5B | 58.4% | 27.5% | 32.7% | 39.5% |
|
||||
| AceGPT-7B-chat | 7B | 69.7% | 27.0% | 35.0% | 43.9% |
|
||||
| ALLaM-7B-Instruct | 7B | 68.5% | 29.0% | 52.2% | 49.9% |
|
||||
| SILMA-9B-Instruct | 9B | 69.7% | 38.0% | 52.9% | 53.5% |
|
||||
|
||||
**Among 0.5B models**: best on COPA-ar (+4.5 vs Qwen), best on HellaSwag-ar (+3.5 vs Qwen), best mean (+1.8 vs Qwen, +5.2 vs Falcon).
|
||||
|
||||
**Ties Falcon-H1-1.5B** on COPA-ar (both 58.4%) at one-third the parameters.
|
||||
|
||||
**Recovers 67%** of SILMA-9B mean accuracy at 5.8% of the parameters.
|
||||
|
||||
## Available Formats
|
||||
|
||||
| Format | Size | Speed (tok/s, bs=1, H100) | Use case |
|
||||
|--------|------|---------------------------|----------|
|
||||
| bf16 | 1.04 GB | 82 (HF generate) | Fine-tuning, research |
|
||||
| int8 | 664 MB | -- | Reduced memory inference |
|
||||
| GGUF f16 | 988 MB | 582 | Maximum quality |
|
||||
| **GGUF q8_0** | **525 MB** | **646** | Best speed |
|
||||
| GGUF q5_k_m | 419 MB | 634 | Balanced |
|
||||
| **GGUF q4_k_m** | **398 MB** | **635** | Smallest footprint |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### With Transformers (bf16)
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_id = "RightNowAI/RightNow-Arabic-0.5B-Turbo"
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id, subfolder="bf16")
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id, subfolder="bf16",
|
||||
torch_dtype="bfloat16", device_map="auto"
|
||||
)
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "أنت مساعد ذكي يجيب باللغة العربية الفصحى"},
|
||||
{"role": "user", "content": "ما هي عاصمة المملكة العربية السعودية؟"}
|
||||
]
|
||||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
outputs = model.generate(**inputs, max_new_tokens=256)
|
||||
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
### With llama.cpp (GGUF)
|
||||
|
||||
```bash
|
||||
# Download the q8_0 quantization (best speed)
|
||||
huggingface-cli download RightNowAI/RightNow-Arabic-0.5B-Turbo \
|
||||
gguf/RightNow-Arabic-0.5B-Turbo-q8_0.gguf --local-dir .
|
||||
|
||||
# Run inference
|
||||
./llama-cli -m RightNow-Arabic-0.5B-Turbo-q8_0.gguf \
|
||||
-p "ما هي أكبر مدينة في مصر؟" \
|
||||
-n 128 --temp 0.7
|
||||
```
|
||||
|
||||
## Training Pipeline
|
||||
|
||||
```
|
||||
Qwen2.5-0.5B (494M, 151,665 vocab)
|
||||
|
|
||||
v
|
||||
Tokenizer Surgery (+27,032 Arabic tokens -> 178,697 vocab)
|
||||
- SentencePiece unigram 32k on 12.5 GB Arabic corpus
|
||||
- Mean-subtoken embedding initialization
|
||||
- Fertility: 2.18 -> 1.80 tok/word (-17.3%)
|
||||
|
|
||||
v
|
||||
Continued Pretraining (504M arwiki tokens)
|
||||
- 2,500 steps, 8xH100 SXM5
|
||||
- FSDP _HYBRID_SHARD_ZERO2 + FlashAttention varlen + Liger
|
||||
- Loss: 14.21 -> 1.69 | Wall time: 6h 57m
|
||||
|
|
||||
v
|
||||
Supervised Fine-Tuning (129,116 instructions)
|
||||
- 5 datasets: evol-instruct-arabic, alpaca-gpt4-arabic,
|
||||
sharegpt-arabic, CIDAR, aya_dataset
|
||||
- Response-only loss masking (72.1% of tokens carry loss)
|
||||
- 5 epochs, 418 steps | Wall time: 12m
|
||||
|
|
||||
v
|
||||
Direct Preference Optimization (6,750 pairs)
|
||||
- argilla-dpo-mix-7k-arabic
|
||||
- 2 epochs, 844 steps | Wall time: 34m
|
||||
|
|
||||
v
|
||||
Weight Soup Merging
|
||||
- Linear(DPO 0.5, SFT 0.25, Pretrain 0.25)
|
||||
- +0.44 points mean accuracy over DPO alone
|
||||
|
|
||||
v
|
||||
Export: bf16, int8, GGUF {f16, q8_0, q5_k_m, q4_k_m}
|
||||
```
|
||||
|
||||
## Training Data
|
||||
|
||||
| Dataset | Examples/Tokens | Use |
|
||||
|---------|----------------|-----|
|
||||
| Arabic Wikipedia (wikimedia/wikipedia 20231101.ar) | 504M tokens | Continued pretraining |
|
||||
| FreedomIntelligence/evol-instruct-arabic | 59,022 | SFT |
|
||||
| FreedomIntelligence/alpaca-gpt4-arabic | 49,969 | SFT |
|
||||
| FreedomIntelligence/sharegpt-arabic | 5,231 | SFT |
|
||||
| arbml/CIDAR | 10,000 | SFT |
|
||||
| CohereForAI/aya_dataset (Arabic) | 4,947 | SFT |
|
||||
| 2A2I/argilla-dpo-mix-7k-arabic | 6,750 | DPO |
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Knowledge ceiling**: At 518M parameters, ArabicMMLU-style knowledge tasks lag 7B+ models by 12-30 points. This is a parameter-count limit, not a training limit.
|
||||
- **MSA only**: Trained on Wikipedia (Modern Standard Arabic). Dialects (Egyptian, Gulf, Levantine) get MSA responses.
|
||||
- **504M pretraining tokens**: Below Chinchilla-optimal ratio. More Arabic data would improve knowledge tasks.
|
||||
- **DPO was weak**: 6,750 machine-translated preference pairs provided minimal signal at 0.5B scale. The weight soup merge was more impactful.
|
||||
- **GGUF tile alignment**: q4_k_m and q5_k_m fall back to higher-bit quantization for 144/290 tensors due to the expanded vocabulary not aligning with k-quant tile sizes.
|
||||
|
||||
## Hardware
|
||||
|
||||
All training ran on a single Nebius `gpu-h100-sxm` node:
|
||||
- 8x NVIDIA H100 80 GB SXM5 HBM3, NVLink4
|
||||
- 128 vCPUs, 1.5 TiB RAM
|
||||
- CUDA 13.0, PyTorch 2.11, flash-attn 2.8.3, transformers 5.5.0
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@article{jaber2025rightnow,
|
||||
title={RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment},
|
||||
author={Jaber, Jaber and Jaber, Osama},
|
||||
year={2025},
|
||||
url={https://huggingface.co/RightNowAI/RightNow-Arabic-0.5B-Turbo}
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Apache 2.0 (same as the base Qwen2.5-0.5B model).
|
||||
|
||||
---
|
||||
|
||||
*Built by [RightNow AI](https://rightnowai.co)*
|
||||
Reference in New Issue
Block a user