TURKCELL/Turkcell-LLM-7b-v1
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
seyma erdem
2024-04-04 11:03:44 +00:00
committed by system
parent 5363d8e1d2
commit aab72cee68
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -9,14 +9,14 @@ alt="Turkcell LLM" width="300"/>
# Turkcell-LLM-7b-v1 # Turkcell-LLM-7b-v1
This model is an extended version of a Mistral-based Large Language Model (LLM) for Turkish. It was trained on a cleaned Turkish raw dataset containing 5 billion tokens. The training process involved using the DORA method followed by fine-tuning with the LORA method. This model is an extended version of a Mistral-based Large Language Model (LLM) for Turkish. It was trained on a cleaned Turkish raw dataset containing 5 billion tokens. The training process involved using the DORA method initially. Following this, we utilized Turkish instruction sets created from various open-source and internal resources for fine-tuning with the LORA method.
## Model Details ## Model Details
- **Base Model**: Mistral 7B based LLM - **Base Model**: Mistral 7B based LLM
- **Tokenizer Extension**: Specifically extended for Turkish - **Tokenizer Extension**: Specifically extended for Turkish
- **Training Dataset**: Cleaned Turkish raw data with 5 billion tokens - **Training Dataset**: Cleaned Turkish raw data with 5 billion tokens
- **Training Method**: Initially with DORA, followed by fine-tuning with LORA - **Training Method**: Initially with DORA, followed by fine-tuning with LORA using custom Turkish instruction sets
### DORA Configuration ### DORA Configuration