193 lines
7.5 KiB
Markdown
193 lines
7.5 KiB
Markdown
---
|
||
library_name: transformers
|
||
tags:
|
||
- gemma2
|
||
- instruct
|
||
- mamaylm
|
||
- insait
|
||
license: gemma
|
||
language:
|
||
- uk
|
||
- en
|
||
base_model:
|
||
- google/gemma-2-9b-it
|
||
- google/gemma-2-9b
|
||
pipeline_tag: text-generation
|
||
---
|
||
# INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1
|
||
|
||
|
||
|
||
INSAIT introduces **MamayLM-Gemma-2-9B-IT-v0.1**, the best performing Ukrainian language model based on **google/gemma-2-9b** and **google/gemma-2-9b-it**.
|
||
MamayLM-Gemma-2-9B-IT-v0.1 is **free to use** and distributed under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).
|
||
This model was created by [`INSAIT`](https://insait.ai/), part of Sofia University St. Kliment Ohridski, in Sofia, Bulgaria.
|
||
|
||
# Model description
|
||
|
||
The model was built on top of Google’s Gemma 2 9B open models.
|
||
It was continuously pre-trained on a large pre-filtered dataset (75B tokens of Ukrainian and English data in total) using the combination of data mixing and model merging,
|
||
allowing the model to gain outstanding Ukrainian cultural and linguistic capabilities while retaining its English performance.
|
||
During the pre-training stage, we use various datasets, including Ukrainian web crawl data (FineWeb2), freely available datasets such as Wikipedia, a range of specialized Ukrainian datasets, and machine translations of popular English datasets.
|
||
The model was then instruction-fine-tuned on a newly constructed Ukrainian instruction dataset created using machine translations of current best English datasets and specialized Ukrainian datasets, prepared by Ukrainian community.
|
||
For more information check our blogpost ([English](https://huggingface.co/blog/INSAIT-Institute/mamaylm), [Ukrainian](https://huggingface.co/blog/INSAIT-Institute/mamaylm-ukr)).
|
||
|
||
# Benchmarks and Results
|
||
|
||
|
||
|
||
|
||
|
||
We evaluate our models on a set of standard English benchmarks, a translated version of them in Ukrainian, as well as, Ukrainian specific benchmarks we collected:
|
||
|
||
- **Winogrande challenge**: testing world knowledge and understanding
|
||
- **Hellaswag**: testing sentence completion
|
||
- **ARC Easy/Challenge**: testing logical reasoning
|
||
- **TriviaQA**: testing trivia knowledge
|
||
- **GSM-8k**: solving multiple-choice questions in high-school mathematics
|
||
- **MMLU**: testing knowledge on a multitude of topics
|
||
- **IFEval**: testing instruction-following skills
|
||
- **ZNO**: testing knowledge of the Ukrainian high school curriculum in Ukrainian language & literature, history, mathematics and geography
|
||
|
||
|
||
These benchmarks test logical reasoning, mathematics, knowledge, language understanding and other skills of the models and are provided at https://github.com/insait-institute/lm-evaluation-harness-uk.
|
||
The graphs above show the performance of MamayLM 9B compared to other large open models. The results show the excellent abilities of MamayLM in Ukrainian, which allow them to **outperform much larger models**,
|
||
including Alibaba’s Qwen 2.5 72B and Meta’s Llama3.1 70B.
|
||
Finally, our models retain the **excellent English performance** inherited from the original Google Gemma 2 models upon which they are based.
|
||
|
||
|
||
|
||
# Use in 🤗 Transformers
|
||
First install the latest version of the transformers library:
|
||
```
|
||
pip install -U 'transformers[torch]'
|
||
```
|
||
Then load the model in transformers:
|
||
```python
|
||
from transformers import AutoModelForCausalLM
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
"INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1",
|
||
torch_dtype=torch.bfloat16,
|
||
attn_implementation="flash_attention_2",
|
||
device_map="auto",
|
||
)
|
||
```
|
||
|
||
# Recommended Parameters
|
||
|
||
For optimal performance, we recommend the following parameters for text generation, as we have extensively tested our model with them:
|
||
|
||
```python
|
||
from transformers import GenerationConfig
|
||
generation_params = GenerationConfig(
|
||
max_new_tokens=2048, # Choose maximum generation tokens
|
||
temperature=0.1,
|
||
top_k=25,
|
||
top_p=1,
|
||
repetition_penalty=1.1,
|
||
eos_token_id=[1,107],
|
||
do_sample=True
|
||
)
|
||
```
|
||
|
||
In principle, increasing temperature should work adequately as well.
|
||
|
||
# Instruction format
|
||
|
||
In order to leverage instruction fine-tuning, your prompt should begin with a beginning-of-sequence token `<bos>` and be formatted in the Gemma 2 chat template. `<bos>` should only be the first token in a chat sequence.
|
||
|
||
E.g.
|
||
```
|
||
<bos><start_of_turn>user
|
||
Хто такий Козак Мамай?<end_of_turn>
|
||
<start_of_turn>model
|
||
|
||
```
|
||
|
||
This format is also available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method:
|
||
|
||
```python
|
||
tokenizer = AutoTokenizer.from_pretrained(
|
||
"INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1",
|
||
use_default_system_prompt=False,
|
||
)
|
||
messages = [
|
||
{"role": "user", "content": "Хто такий Козак Мамай?"},
|
||
]
|
||
input_ids = tokenizer.apply_chat_template(
|
||
messages,
|
||
return_tensors="pt",
|
||
add_generation_prompt=True,
|
||
return_dict=True
|
||
)
|
||
outputs = model.generate(
|
||
**input_ids,
|
||
generation_config=generation_params
|
||
)
|
||
print(tokenizer.decode(outputs[0]))
|
||
```
|
||
|
||
# Use with vLLM
|
||
|
||
Example usage with vLLM:
|
||
|
||
```python
|
||
from vllm import LLM, SamplingParams
|
||
from vllm.inputs import TokensPrompt
|
||
from transformers import AutoTokenizer
|
||
tokenizer = AutoTokenizer.from_pretrained(
|
||
"INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1",
|
||
use_default_system_prompt=False,
|
||
)
|
||
sampling_params = SamplingParams(
|
||
max_tokens=2048,
|
||
temperature=0.1,
|
||
top_k=25,
|
||
top_p=1,
|
||
repetition_penalty=1.1,
|
||
stop_token_ids=[1, 107],
|
||
)
|
||
llm = LLM(
|
||
model="INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1",
|
||
dtype="bfloat16",
|
||
# enforce_eager=True
|
||
)
|
||
messages = [
|
||
{"role": "user", "content": "Хто такий Козак Мамай?"},
|
||
]
|
||
formatted_prompt = tokenizer.apply_chat_template(
|
||
messages,
|
||
tokenize=False,
|
||
add_generation_prompt=True
|
||
)
|
||
input_ids = tokenizer(
|
||
formatted_prompt,
|
||
add_special_tokens=False
|
||
).input_ids
|
||
prompt = TokensPrompt(prompt_token_ids=input_ids)
|
||
output = llm.generate(
|
||
prompt,
|
||
sampling_params
|
||
)
|
||
generated_text = output[0].outputs[0].text
|
||
print(generated_text)
|
||
```
|
||
|
||
# Use with GGML / llama.cpp
|
||
|
||
The model and instructions for usage in GGUF format are available at [INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1-GGUF](https://huggingface.co/INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1-GGUF).
|
||
|
||
# Community Feedback
|
||
|
||
We welcome feedback from the community to help improve MamayLM. If you have suggestions, encounter any issues, or have ideas for improvements, please:
|
||
- Share your experience using the model through Hugging Face's community discussion feature or
|
||
- Contact us at [contact@insait.ai](mailto:contact@insait.ai)
|
||
|
||
Your real-world usage and insights are valuable in helping us optimize the model's performance and behaviour for various use cases.
|
||
|
||
# Summary
|
||
- **Finetuned from:** [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it); [google/gemma-2-9b](https://huggingface.co/google/gemma-2-9b);
|
||
- **Model type:** Causal decoder-only transformer language model
|
||
- **Language:** Ukrainian and English
|
||
- **Contact:** [contact@insait.ai](mailto:contact@insait.ai)
|
||
- **License:** MamayLM is distributed under [Gemma Terms of Use](https://huggingface.co/INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1/raw/main/LICENSE)
|