151 lines
7.4 KiB
Markdown
151 lines
7.4 KiB
Markdown
|
---
|
||
|
|
license: cc-by-nc-sa-4.0
|
||
|
|
language:
|
||
|
|
- multilingual
|
||
|
|
- fa
|
||
|
|
- en
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- text-generation-inference
|
||
|
|
inference: false
|
||
|
|
metrics:
|
||
|
|
- bleu
|
||
|
|
- comet
|
||
|
|
- accuracy
|
||
|
|
- perplexity
|
||
|
|
- spearmanr
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
co2_eq_emissions:
|
||
|
|
emissions: 232380
|
||
|
|
source: "PersianMind: A Cross-Lingual Persian-English Large Language Model. https://arxiv.org/abs/2401.06466"
|
||
|
|
training_type: "fine-tuning"
|
||
|
|
hardware_used: "4 RTX3090 24GB GPUs"
|
||
|
|
geographical_location: "Tehran, Iran"
|
||
|
|
---
|
||
|
|
|
||
|
|
<p align="center">
|
||
|
|
<img src="PersianMind.jpg" alt="PersianMind logo" width=200/>
|
||
|
|
</p>
|
||
|
|
|
||
|
|
|
||
|
|
# <span style="font-variant:small-caps;">PersianMind</span>
|
||
|
|
|
||
|
|
<span style="font-variant:small-caps;">PersianMind</span> is a cross-lingual Persian-English large language model.
|
||
|
|
The model achieves state-of-the-art results on Persian subset of the [<span style="font-variant:small-caps;">Belebele</span>](https://github.com/facebookresearch/belebele) benchmark
|
||
|
|
and the [ParsiNLU multiple-choice QA](https://github.com/persiannlp/parsinlu) task.
|
||
|
|
It also attains performance comparable to GPT-3.5-turbo in a Persian reading comprehension task.
|
||
|
|
|
||
|
|
## Model Description
|
||
|
|
|
||
|
|
- **Developed by:** [Pedram Rostami](mailto:pedram.rostami@ut.ac.ir), [Ali Salemi](mailto:alisalemi@ut.ac.ir), and [Mohammad Javad Dousti](mailto:mjdousti@ut.ac.ir)
|
||
|
|
- **Model type:** Language model
|
||
|
|
- **Languages:** English and Persian
|
||
|
|
- **License:** [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) (non-commercial use only.)
|
||
|
|
|
||
|
|
## How to Get Started with the Model
|
||
|
|
|
||
|
|
Use the code below to get started with the model.
|
||
|
|
Note that you need to install <code><b>sentencepiece</b></code> and <code><b>accelerate</b></code> libraries along with <code><b>PyTorch</b></code> and <code><b>🤗Transformers</b></code> to run this code.
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
|
import torch
|
||
|
|
|
||
|
|
device = "cuda" if torch.cuda.is_available() else "cpu"
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
"universitytehran/PersianMind-v1.0",
|
||
|
|
torch_dtype=torch.bfloat16,
|
||
|
|
low_cpu_mem_usage=True,
|
||
|
|
device_map={"": device},
|
||
|
|
)
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(
|
||
|
|
"universitytehran/PersianMind-v1.0",
|
||
|
|
)
|
||
|
|
|
||
|
|
TEMPLATE = "{context}\nYou: {prompt}\nPersianMind: "
|
||
|
|
CONTEXT = "This is a conversation with PersianMind. It is an artificial intelligence model designed by a team of " \
|
||
|
|
"NLP experts at the University of Tehran to help you with various tasks such as answering questions, " \
|
||
|
|
"providing recommendations, and helping with decision making. You can ask it anything you want and " \
|
||
|
|
"it will do its best to give you accurate and relevant information."
|
||
|
|
PROMPT = "در مورد هوش مصنوعی توضیح بده."
|
||
|
|
|
||
|
|
model_input = TEMPLATE.format(context=CONTEXT, prompt=PROMPT)
|
||
|
|
input_tokens = tokenizer(model_input, return_tensors="pt")
|
||
|
|
input_tokens = input_tokens.to(device)
|
||
|
|
generate_ids = model.generate(**input_tokens, max_new_tokens=512, do_sample=False, repetition_penalty=1.1)
|
||
|
|
model_output = tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
|
||
|
|
|
||
|
|
print(model_output[len(model_input):])
|
||
|
|
```
|
||
|
|
|
||
|
|
### How to Quantize the Model
|
||
|
|
|
||
|
|
Quantized models can be run on resource-constrained devices.
|
||
|
|
To quantize the model, you should install the <code><b>bitsandbytes</b></code> library.
|
||
|
|
In order to quantize the model in 8-bit (`INT8`), use the code below.
|
||
|
|
|
||
|
|
```python
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
"universitytehran/PersianMind-v1.0",
|
||
|
|
device_map="auto",
|
||
|
|
low_cpu_mem_usage=True,
|
||
|
|
load_in_8bit=True
|
||
|
|
)
|
||
|
|
```
|
||
|
|
|
||
|
|
Alternatively, you can quantize the model in 4-bit (`NormalFloat4`) with the following code.
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import BitsAndBytesConfig
|
||
|
|
|
||
|
|
quantization_config = BitsAndBytesConfig(
|
||
|
|
load_in_4bit=True,
|
||
|
|
bnb_4bit_use_double_quant=True,
|
||
|
|
bnb_4bit_quant_type="nf4",
|
||
|
|
)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
"universitytehran/PersianMind-v1.0",
|
||
|
|
quantization_config=quantization_config,
|
||
|
|
device_map="auto"
|
||
|
|
)
|
||
|
|
```
|
||
|
|
|
||
|
|
### Evaluating Quantized Models
|
||
|
|
|
||
|
|
| Model | <span style="font-variant:small-caps;">Belebele</span> (Persian) | Fa→En Translation<br>(<span style="font-variant:small-caps;">Comet</span>) | En→Fa Translation<br>(<span style="font-variant:small-caps;">Comet</span>) | Model Size | Tokens/sec |
|
||
|
|
| :----------------------------------------------------------------: | :--------------------------------------------------------------: | :------------------------------------------------------------------------: | :------------------------------------------------------------------------: | :--------: | :--------: |
|
||
|
|
| <span style="font-variant:small-caps;">PersianMind</span> (`BF16`) | 73.9 | 83.61 | 79.44 | 13.7G | 25.35 |
|
||
|
|
| <span style="font-variant:small-caps;">PersianMind</span> (`INT8`) | 73.7 | 82.32 | 78.61 | 7.2G | 11.36 |
|
||
|
|
| <span style="font-variant:small-caps;">PersianMind</span> (`NF4`) | 70.2 | 82.07 | 80.36 | 3.9G | 24.36 |
|
||
|
|
|
||
|
|
We evaluated quantized models in various tasks against the original model.
|
||
|
|
Specifically, we evaluated all models using the reading comprehension multiple-choice
|
||
|
|
question-answering benchmark of [<span style="font-variant:small-caps;">Belebele</span>](https://github.com/facebookresearch/belebele) (Persian subset) and reported the accuracy of each model.
|
||
|
|
Additionally, we evaluated our models for Persian-to-English and English-to-Persian translation tasks.
|
||
|
|
For this, we utilized the Persian-English subset of the [<span style="font-variant:small-caps;">Flores</span>-200](https://github.com/facebookresearch/flores/tree/main/flores200) dataset and
|
||
|
|
reported our results using the <span style="font-variant:small-caps;">Comet</span> metric.
|
||
|
|
Furthermore, we calculated the average number of generated tokens per second by each model during running the translation tasks.
|
||
|
|
To understand resource efficiency, we measured the memory usage of each model by employing the `get_memory_footprint()` function.
|
||
|
|
|
||
|
|
## License
|
||
|
|
<span style="font-variant:small-caps;">PersianMind</span> is subject to Meta's [LLaMa2 Community License](https://raw.githubusercontent.com/facebookresearch/llama/main/LICENSE).
|
||
|
|
It is further licensed under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/), which allows non-commercial use of the model.
|
||
|
|
Commercial use of this model requires written agreement which must be obtained from the copyright holders who are listed as developers in this page.
|
||
|
|
If you suspect any violations, please reach out to us.
|
||
|
|
|
||
|
|
|
||
|
|
## Citation
|
||
|
|
|
||
|
|
If you find this model helpful, please ensure to cite the following paper.
|
||
|
|
|
||
|
|
**BibTeX:**
|
||
|
|
```bibtex
|
||
|
|
@misc{persianmind,
|
||
|
|
title={{PersianMind: A Cross-Lingual Persian-English Large Language Model}},
|
||
|
|
author={Rostami, Pedram and Salemi, Ali and Dousti, Mohammad Javad},
|
||
|
|
year={2024}
|
||
|
|
eprint={2401.06466},
|
||
|
|
archivePrefix={arXiv},
|
||
|
|
primaryClass={cs.CL}
|
||
|
|
}
|
||
|
|
```
|