108 lines
3.7 KiB
Markdown
108 lines
3.7 KiB
Markdown
---
|
|
base_model: unsloth/Meta-Llama-3.1-8B
|
|
tags:
|
|
- text-generation-inference
|
|
- transformers
|
|
- unsloth
|
|
- llama
|
|
- trl
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
- ur
|
|
---
|
|
|
|
|
|
# Model Card for Alif 1.0 8B Instruct
|
|
|
|
**Alif 1.0 8B Instruct** is an open-source model with highly advanced multilingual reasoning capabilities. It utilizes human refined multilingual synthetic data paired with reasoning to enhance cultural nuance and reasoning capabilities in english and urdu languages.
|
|
|
|
- **Developed by:** large-traversaal
|
|
- **License:** apache-2.0
|
|
- **Base model:** unsloth/Meta-Llama-3.1-8B
|
|
- **Model:** Alif-1.0-8B-Instruct
|
|
- **Model Size:** 8 billion parameters
|
|
|
|
This model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
|
|
|
|
|
### How to Use Alif 1.0 8B Instruct
|
|
|
|
Install the transformers, bitsandbytes libraries and load Alif 1.0 8B Instruct as follows:
|
|
|
|
```python
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
|
|
import torch
|
|
from transformers import BitsAndBytesConfig
|
|
|
|
model_id = "large-traversaal/Alif-1.0-8B-Instruct"
|
|
|
|
# 4-bit quantization configuration
|
|
quantization_config = BitsAndBytesConfig(
|
|
load_in_4bit=True,
|
|
bnb_4bit_compute_dtype=torch.float16,
|
|
bnb_4bit_use_double_quant=True,
|
|
bnb_4bit_quant_type="nf4"
|
|
)
|
|
|
|
# Load tokenizer and model in 4-bit
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_id,
|
|
quantization_config=quantization_config,
|
|
device_map="auto"
|
|
)
|
|
|
|
# Create text generation pipeline
|
|
chatbot = pipeline("text-generation", model=model, tokenizer=tokenizer, device_map="auto")
|
|
|
|
# Function to chat
|
|
def chat(message):
|
|
response = chatbot(message, max_new_tokens=100, do_sample=True, temperature=0.3)
|
|
return response[0]["generated_text"]
|
|
|
|
# Example chat
|
|
user_input = "شہر کراچی کی کیا اہمیت ہے؟"
|
|
bot_response = chat(user_input)
|
|
|
|
print(bot_response)
|
|
|
|
```
|
|
|
|
You can also try out this model using [TextStreamer](https://colab.research.google.com/drive/1mEPynC__uN2tKDvDho3f6MpcKW-GMiAh?usp=sharing) or [Gradio](https://colab.research.google.com/drive/1DUwlYBOMUd7FZaI631-y6y8fTNiy0pqt?usp=sharing) in Colab. It is also available in GGUF with various quantized formats for Ollama, LM Studio, Jan, and Llama.cpp.
|
|
|
|
|
|
## Model Details
|
|
|
|
**Input**: Models input text only.
|
|
|
|
**Output**: Models generate text only.
|
|
|
|
**Model Architecture**: Alif 1.0 8B Instruct is an auto-regressive language model that uses an optimized transformer architecture. Post-training includes continuous pretraining and supervised finetuning.
|
|
|
|
For more details about how the model was trained, check out [our blogpost](https://blog.traversaal.ai/announcing-alif-1-0-our-first-urdu-llm-outperforming-other-open-source-llms/).
|
|
|
|
|
|
### Evaluation
|
|
We evaluated Alif 1.0 8B Instruct against Gemma 2 9B, Llama 3.1 8B, Mistral Nemo 12B, Qwen 2.5 7B and Cohere Aya Expanse 8B using the human annotated Urdu evaluation dataset and scores are determined using gpt-4o as a judge.
|
|
|
|
<img src="result1.jpg" width="650" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
|
|
|
<img src="result2.jpg" width="650" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
|
|
|
### Citation
|
|
```bash
|
|
@article{ShafiqueAlif2025,
|
|
title = {Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation},
|
|
author = {Muhammad Ali Shafique and Kanwal Mehreen and Muhammad Arham and Maaz Amjad and Sabur Butt and Hamza Farooq},
|
|
journal = {arXiv preprint arXiv:2510.09051},
|
|
year = {2025},
|
|
url = {https://arxiv.org/abs/2510.09051}
|
|
}
|
|
```
|
|
|
|
### Model Card Contact
|
|
|
|
For errors or additional questions about details in this model card, contact: contact@traversaal.ai
|
|
|