初始化项目,由ModelHub XC社区提供模型
Model: AhiskaAI/AhiskaAI-308m-Instruct-v0.2 Source: Original Platform
This commit is contained in:
133
README.md
Normal file
133
README.md
Normal file
@@ -0,0 +1,133 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- tr
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- instruct
|
||||
- chat
|
||||
- conversational
|
||||
- fine-tuned
|
||||
- slm
|
||||
- llama
|
||||
- tr-llm
|
||||
- Ahiska
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
|
||||
# AhiskaAI-308m-IT-v0.2
|
||||
|
||||
AhiskaAI-308m-IT-v0.2 is the instruction-tuned version of our 308M parameter Small Language Model. Fine-tuned on 16,000+ curated Turkish instruction-response pairs, it is designed to provide stronger conversational ability and improved instruction following while remaining efficient enough to run on consumer hardware.
|
||||
|
||||
**Base Model:** AhiskaAI-308m-Base-v0.2
|
||||
|
||||
---
|
||||
|
||||
## Model Details
|
||||
|
||||
- **Architecture:** Llama-based architecture.
|
||||
- **Fine-tuning:** Supervised Fine-Tuning (SFT).
|
||||
- **Format:** ChatML.
|
||||
- **Parameters:** 308M.
|
||||
- **Context Window:** 1024 tokens.
|
||||
- **Tokenizer:** Custom BPE Tokenizer (Vocabulary Size: 32,000).
|
||||
- **Training Framework:** PyTorch & Transformers.
|
||||
- **Hardware:** NVIDIA RTX 4050 6GB Laptop GPU.
|
||||
|
||||
---
|
||||
|
||||
## Fine-tuning Dataset
|
||||
|
||||
The model was fine-tuned using more than **16,000** carefully curated Turkish instruction-response pairs.
|
||||
|
||||
The dataset includes tasks such as:
|
||||
|
||||
- Question answering
|
||||
- General conversation
|
||||
- Summarization
|
||||
- Text generation
|
||||
- Instruction following
|
||||
- Basic reasoning
|
||||
|
||||
---
|
||||
|
||||
## Design Goal
|
||||
|
||||
The 308M-IT model serves as the flagship conversational model of the AhiskaAI v0.2 family.
|
||||
|
||||
Its primary objectives are:
|
||||
|
||||
- Improved Turkish instruction following.
|
||||
- Better contextual understanding.
|
||||
- More natural conversational responses.
|
||||
- A strong research foundation for future preference alignment methods such as DPO.
|
||||
|
||||
---
|
||||
|
||||
## Training Logs
|
||||
|
||||

|
||||
|
||||
*The graph above demonstrates the supervised fine-tuning convergence of AhiskaAI-308m-IT-v0.2.*
|
||||
|
||||
---
|
||||
|
||||
## Usage (ChatML Format)
|
||||
|
||||
### Recommended System Prompt
|
||||
|
||||
```
|
||||
Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın.
|
||||
```
|
||||
|
||||
### Example Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2")
|
||||
tokenizer = AutoTokenizer.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2")
|
||||
|
||||
SYSTEM_PROMPT = "Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın."
|
||||
|
||||
user_query = "Ahıska Türkleri hakkında bilgi verir misin?"
|
||||
|
||||
prompt = (
|
||||
f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n"
|
||||
f"<|im_start|>user\n{user_query}<|im_end|>\n"
|
||||
f"<|im_start|>assistant\n"
|
||||
)
|
||||
|
||||
inputs = tokenizer(prompt, return_tensors="pt")
|
||||
|
||||
outputs = model.generate(**inputs, max_new_tokens=200)
|
||||
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known Limitations
|
||||
|
||||
- Optimized primarily for Turkish.
|
||||
- Context window is limited to 1024 tokens.
|
||||
- Factual accuracy is still limited by model size and pretraining data.
|
||||
- May generate incorrect or incomplete responses on complex reasoning tasks.
|
||||
|
||||
---
|
||||
|
||||
## Future Plans
|
||||
|
||||
- Preference alignment using DPO.
|
||||
- Larger, higher-quality Turkish datasets.
|
||||
- Expanded evaluation benchmarks.
|
||||
- Future AhiskaAI v0.3 model family.
|
||||
|
||||
---
|
||||
|
||||
## About AhiskaAI
|
||||
|
||||
AhiskaAI is an independent open-source initiative dedicated to developing efficient Turkish Small Language Models trained completely from scratch.
|
||||
|
||||
Follow us on Hugging Face for updates and future releases.
|
||||
Reference in New Issue
Block a user