AhiskaAI-308m-Instruct-v0.2/README.md

---
license: apache-2.0
language:
- tr
pipeline_tag: text-generation
tags:
- instruct
- chat
- conversational
- fine-tuned
- slm
- llama
- tr-llm
- Ahiska
library_name: transformers
---


# AhiskaAI-308m-IT-v0.2

AhiskaAI-308m-IT-v0.2 is the instruction-tuned version of our 308M parameter Small Language Model. Fine-tuned on 16,000+ curated Turkish instruction-response pairs, it is designed to provide stronger conversational ability and improved instruction following while remaining efficient enough to run on consumer hardware.

**Base Model:** AhiskaAI-308m-Base-v0.2

---

## Model Details

- **Architecture:** Llama-based architecture.
- **Fine-tuning:** Supervised Fine-Tuning (SFT).
- **Format:** ChatML.
- **Parameters:** 308M.
- **Context Window:** 1024 tokens.
- **Tokenizer:** Custom BPE Tokenizer (Vocabulary Size: 32,000).
- **Training Framework:** PyTorch & Transformers.
- **Hardware:** NVIDIA RTX 4050 6GB Laptop GPU.

---

## Fine-tuning Dataset

The model was fine-tuned using more than **16,000** carefully curated Turkish instruction-response pairs.

The dataset includes tasks such as:

- Question answering
- General conversation
- Summarization
- Text generation
- Instruction following
- Basic reasoning

---

## Design Goal

The 308M-IT model serves as the flagship conversational model of the AhiskaAI v0.2 family.

Its primary objectives are:

- Improved Turkish instruction following.
- Better contextual understanding.
- More natural conversational responses.
- A strong research foundation for future preference alignment methods such as DPO.

---

## Training Logs

![Training Loss Curve](training_loss.png)

*The graph above demonstrates the supervised fine-tuning convergence of AhiskaAI-308m-IT-v0.2.*

---

## Usage (ChatML Format)

### Recommended System Prompt

```
Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın.
```

### Example Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2")
tokenizer = AutoTokenizer.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2")

SYSTEM_PROMPT = "Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın."

user_query = "Ahıska Türkleri hakkında bilgi verir misin?"

prompt = (
    f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n"
    f"<|im_start|>user\n{user_query}<|im_end|>\n"
    f"<|im_start|>assistant\n"
)

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## Known Limitations

- Optimized primarily for Turkish.
- Context window is limited to 1024 tokens.
- Factual accuracy is still limited by model size and pretraining data.
- May generate incorrect or incomplete responses on complex reasoning tasks.

---

## Future Plans

- Preference alignment using DPO.
- Larger, higher-quality Turkish datasets.
- Expanded evaluation benchmarks.
- Future AhiskaAI v0.3 model family.

---

## About AhiskaAI

AhiskaAI is an independent open-source initiative dedicated to developing efficient Turkish Small Language Models trained completely from scratch.

Follow us on Hugging Face for updates and future releases.