215 lines
6.8 KiB
Markdown
215 lines
6.8 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
language:
|
|||
|
|
- ar
|
|||
|
|
- arz
|
|||
|
|
library_name: transformers
|
|||
|
|
pipeline_tag: automatic-speech-recognition
|
|||
|
|
datasets:
|
|||
|
|
- YouTube
|
|||
|
|
- rsalshalan/MGB3
|
|||
|
|
- pain/MASC
|
|||
|
|
- mozilla-foundation/common_voice_15_0
|
|||
|
|
- halabi2016/arabic_speech_corpus
|
|||
|
|
model-index:
|
|||
|
|
- name: egyptian-arabic-wav2vec2-xlsr-53
|
|||
|
|
results:
|
|||
|
|
- task:
|
|||
|
|
name: Speech Recognition
|
|||
|
|
type: automatic-speech-recognition
|
|||
|
|
dataset:
|
|||
|
|
name: mozilla-foundation/common_voice_17_0
|
|||
|
|
type: mozilla-foundation/common_voice_17_0
|
|||
|
|
args: ar
|
|||
|
|
metrics:
|
|||
|
|
- name: Test WER
|
|||
|
|
type: wer
|
|||
|
|
value: 27.20
|
|||
|
|
base_model:
|
|||
|
|
- omarxadel/wav2vec2-large-xlsr-53-arabic-egyptian
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# 🐪🇪🇬 Egyptian Arabic ASR — wav2vec2-large-xlsr-53 Fine-tuned
|
|||
|
|
|
|||
|
|
This model is a fine-tuned version of [omarxadel/wav2vec2-large-xlsr-53-arabic-egyptian](https://huggingface.co/omarxadel/wav2vec2-large-xlsr-53-arabic-egyptian),
|
|||
|
|
enhancing **Egyptian Arabic**, **Modern Standard Arabic (MSA)** and **Gulf / Levantine Arabic** for Automatic Speech Recognition.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📚 Dataset
|
|||
|
|
|
|||
|
|
It was trained on a diverse combination of publicly available and custom-collected Arabic speech datasets, including:
|
|||
|
|
|
|||
|
|
- **📺 YouTube Egyptian Arabic Speech** *(custom-curated)*
|
|||
|
|
- **🎧 MASC** *(Media Arabic Speech Corpus)*
|
|||
|
|
- **🌍 Common Voice 15 - Arabic**
|
|||
|
|
- **📻 MGB-3 Broadcast Speech**
|
|||
|
|
- **🗂️ Arabic Speech Corpus**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🔥 Model Highlights
|
|||
|
|
|
|||
|
|
- 📌 Focused on real-life Egyptian Arabic speech (YouTube, spontaneous, conversational)
|
|||
|
|
- 🚀 Supports MSA and other Arabic dialects.
|
|||
|
|
- 🔉 Trained on both scripted and natural speech
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 💬 Languages & Dialects
|
|||
|
|
|
|||
|
|
| Dialect | Coverage |
|
|||
|
|
| ---------------------------- | ------------ |
|
|||
|
|
| Egyptian Arabic | ✅ Primary |
|
|||
|
|
| Modern Standard Arabic (MSA) | ✅ Supported |
|
|||
|
|
| Gulf / Levantine | ✅ Supported |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🚀 Usage
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import pipeline
|
|||
|
|
|
|||
|
|
asr = pipeline("automatic-speech-recognition", model="IbrahimAmin/egyptian-arabic-wav2vec2-xlsr-53")
|
|||
|
|
asr("path/to/audio.wav")
|
|||
|
|
|
|||
|
|
# Long-Form Transcription: https://huggingface.co/blog/asr-chunking
|
|||
|
|
asr = pipeline("automatic-speech-recognition", model="IbrahimAmin/egyptian-arabic-wav2vec2-xlsr-53", chunk_length_s=30)
|
|||
|
|
asr("path/to/audio.wav")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
|
|||
|
|
import torch
|
|||
|
|
import torchaudio
|
|||
|
|
|
|||
|
|
model = Wav2Vec2ForCTC.from_pretrained("IbrahimAmin/egyptian-arabic-wav2vec2-xlsr-53")
|
|||
|
|
processor = Wav2Vec2Processor.from_pretrained("IbrahimAmin/egyptian-arabic-wav2vec2-xlsr-53")
|
|||
|
|
|
|||
|
|
# Load audio (must be mono, 16kHz)
|
|||
|
|
waveform, sr = torchaudio.load("path/to/audio.wav")
|
|||
|
|
|
|||
|
|
# Convert to mono if not already
|
|||
|
|
if waveform.shape[0] > 1:
|
|||
|
|
waveform = torch.mean(waveform, dim=0, keepdim=True)
|
|||
|
|
|
|||
|
|
# Resample if needed to 16 kHz
|
|||
|
|
if sr != 16000:
|
|||
|
|
resampler = torchaudio.transforms.Resample(orig_freq=sr, new_freq=16000)
|
|||
|
|
waveform = resampler(waveform)
|
|||
|
|
|
|||
|
|
inputs = processor(waveform.squeeze(), sampling_rate=16000, return_tensors="pt")
|
|||
|
|
|
|||
|
|
with torch.inference_mode():
|
|||
|
|
logits = model(**inputs).logits
|
|||
|
|
|
|||
|
|
predicted_ids = torch.argmax(logits, dim=-1)
|
|||
|
|
transcription = processor.batch_decode(predicted_ids)
|
|||
|
|
print(transcription)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🧪 Evaluation
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
import torch
|
|||
|
|
import torchaudio
|
|||
|
|
import re
|
|||
|
|
from datasets import load_dataset
|
|||
|
|
from evaluate import load
|
|||
|
|
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
|
|||
|
|
|
|||
|
|
# Device setup
|
|||
|
|
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
|
|||
|
|
|
|||
|
|
# 🔑 Replace with your Hugging Face token and the desired Wav2Vec2-based model ID
|
|||
|
|
HF_TOKEN = "your_hf_token"
|
|||
|
|
MODEL_NAME = "your_model_name_or_path"
|
|||
|
|
|
|||
|
|
# Load the Common Voice 17.0 Arabic test split
|
|||
|
|
test_dataset = load_dataset(
|
|||
|
|
"mozilla-foundation/common_voice_17_0",
|
|||
|
|
"ar",
|
|||
|
|
split="test",
|
|||
|
|
token=HF_TOKEN
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
# Load WER metric
|
|||
|
|
wer = load("wer")
|
|||
|
|
|
|||
|
|
# Load processor and model
|
|||
|
|
processor = Wav2Vec2Processor.from_pretrained(MODEL_NAME, token=HF_TOKEN)
|
|||
|
|
model = Wav2Vec2ForCTC.from_pretrained(MODEL_NAME, token=HF_TOKEN).to(device)
|
|||
|
|
|
|||
|
|
# Define regex for cleaning up unwanted characters
|
|||
|
|
CHARS_TO_IGNORE_REGEX = r'[\؛\—\_get\«\»\ـ\,\?\.\!\-\;\:"\“\%\‘\”\<5C>\#\،\☭,\؟]'
|
|||
|
|
|
|||
|
|
def preprocess(batch):
|
|||
|
|
"""Removes unwanted characters and resamples audio to 16kHz."""
|
|||
|
|
batch["sentence"] = re.sub(CHARS_TO_IGNORE_REGEX, "", batch["sentence"])
|
|||
|
|
speech_array, sampling_rate = torchaudio.load(batch["path"])
|
|||
|
|
resampler = torchaudio.transforms.Resample(orig_freq=sampling_rate, new_freq=16_000)
|
|||
|
|
batch["speech"] = resampler(speech_array).squeeze().numpy()
|
|||
|
|
return batch
|
|||
|
|
|
|||
|
|
# Apply preprocessing
|
|||
|
|
test_dataset = test_dataset.map(preprocess)
|
|||
|
|
|
|||
|
|
def predict(batch):
|
|||
|
|
"""Runs inference and decodes predicted text."""
|
|||
|
|
inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
|
|||
|
|
|
|||
|
|
with torch.inference_mode():
|
|||
|
|
logits = model(
|
|||
|
|
input_values=inputs["input_values"].to(device),
|
|||
|
|
attention_mask=inputs["attention_mask"].to(device)
|
|||
|
|
).logits
|
|||
|
|
|
|||
|
|
predicted_ids = torch.argmax(logits, dim=-1)
|
|||
|
|
batch["pred_strings"] = processor.batch_decode(predicted_ids)
|
|||
|
|
return batch
|
|||
|
|
|
|||
|
|
# Run prediction
|
|||
|
|
result = test_dataset.map(predict, batched=True, batch_size=8)
|
|||
|
|
|
|||
|
|
# Compute and print Word Error Rate
|
|||
|
|
wer_score = wer.compute(predictions=result["pred_strings"], references=result["sentence"])
|
|||
|
|
print(f"WER: {wer_score * 100:.2f}%")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🗣️ Model Comparison on Common Voice 17.0 Arabic Subset (Test Set)
|
|||
|
|
|
|||
|
|
| **Model** | **WER (%)** |
|
|||
|
|
| -------------------------------------------------- | ----------: |
|
|||
|
|
| **`IbrahimAmin/egyptian-arabic-wav2vec2-xlsr-53`** | **27.20** |
|
|||
|
|
| `jonatasgrosman/wav2vec2-large-xlsr-53-arabic` | 45.55 |
|
|||
|
|
| `AndrewMcDowell/wav2vec2-xls-r-300m-arabic` | 47.22 |
|
|||
|
|
| `openai/whisper-large-v3`* | 52.36 |
|
|||
|
|
| `Ahmed107/hamsa-v0.6Q`* | 53.27 |
|
|||
|
|
| `nadsoft/hamsa-v0.1-beta`* | 65.60 |
|
|||
|
|
| `openai/whisper-medium`* | 67.75 |
|
|||
|
|
| `openai/whisper-small`* | 74.16 |
|
|||
|
|
| `omarxadel/wav2vec2-large-xlsr-53-arabic-egyptian` | 91.82 |
|
|||
|
|
| `arbml/wav2vec2-large-xlsr-53-arabic-egyptian` | 93.92 |
|
|||
|
|
| `mboushaba/whisper-large-v3-turbo-arabic`* | 96.90 |
|
|||
|
|
|
|||
|
|
\*: *Whisper models were decoded using beam search (`beam_size = 5`) and evaluated using `BasicTextNormalizer` with `remove_diacritics=False` and `split_letters=False`, applied to both predictions and reference text.*
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ✨ Citation
|
|||
|
|
|
|||
|
|
If you want to cite this model you can use this:
|
|||
|
|
|
|||
|
|
```bibtex
|
|||
|
|
@misc{amin2025egyptianasr,
|
|||
|
|
title={Egyptian Arabic ASR with wav2vec2 XLSR 53},
|
|||
|
|
author={Ibrahim Amin},
|
|||
|
|
year={2025},
|
|||
|
|
howpublished={\url{https://huggingface.co/IbrahimAmin/egyptian-arabic-wav2vec2-xlsr-53}},
|
|||
|
|
}
|
|||
|
|
```
|