170 lines
4.4 KiB
Markdown
170 lines
4.4 KiB
Markdown
---
|
||
license: apache-2.0
|
||
language:
|
||
- ar
|
||
- en
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- text-generation
|
||
- pytorch
|
||
- transformers
|
||
- vllm
|
||
- causal-lm
|
||
- depth-extension
|
||
- arabic
|
||
- english
|
||
- karnak
|
||
- qwen
|
||
base_model: Qwen/Qwen3-30B-A3B-Instruct-2507
|
||
model_name: Karnak
|
||
parameters: 40B
|
||
inference: false
|
||
---
|
||
|
||
# Karnak: Enhanced Arabic–English Large Language Model
|
||
|
||
## Model Summary
|
||
|
||
**Karnak** is a depth-extended causal language model optimized for **Arabic and English** generation. It is built on top of **Qwen/Qwen3-30B-A3B-Instruct-2507**, featuring architectural depth extension and a tokenizer specifically optimized for Arabic to improve fluency and efficiency.
|
||
|
||
Karnak was trained using **high-quality, filtered data** through a rigorous pipeline to enhance overall instruction-following capabilities, factuality, and robustness.
|
||
|
||
## Key Features
|
||
|
||
- **Depth Extension (~40B):** Expanded depth to increase reasoning capacity and improve long-range dependency modeling.
|
||
- **Arabic-Optimized Tokenizer:** Improved Arabic tokenization efficiency, resulting in reduced token fragmentation and higher-quality generation.
|
||
- **Multi-Stage Training:** The model evolved through: Pre-trained weights → Depth Extension → Continued Pre-training → SFT (Supervised Fine-Tuning).
|
||
- **Extended Context Window:** Designed for long-context usage with a **safe context range up to 20K tokens** (recommended to stay within this limit for optimal stability).
|
||
|
||
## Model Details
|
||
|
||
- **Model Name:** Karnak
|
||
- **Base Model:** Qwen/Qwen3-30B-A3B-Instruct-2507
|
||
- **Parameter Count:** ~40B (Depth-Extended)
|
||
- **Languages:** Arabic, English
|
||
- **Training:** High-quality filtered data + Multi-stage pipeline (Continued pre-training + SFT)
|
||
- **Safe Context Range:** Up to **20,000 tokens**
|
||
|
||
---
|
||
|
||
## Usage
|
||
|
||
### 1) Hugging Face Transformers
|
||
|
||
To use Karnak with the standard Transformers library, ensure you have the latest version installed.
|
||
|
||
```bash
|
||
pip install -U "transformers>=4.40.0" torch accelerate
|
||
```
|
||
|
||
Python Code Example (Chat Template):
|
||
|
||
```python
|
||
import torch
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model_id = "Applied-Innovation-Center/Karnak"
|
||
|
||
# Load tokenizer and model
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_id,
|
||
device_map="auto",
|
||
torch_dtype=torch.bfloat16,
|
||
trust_remote_code=True,
|
||
)
|
||
|
||
# Prepare Input
|
||
prompt = "اشرح لي نظرية النسبية بشكل مبسط."
|
||
messages = [
|
||
{"role": "system", "content": "You are a helpful assistant."},
|
||
{"role": "user", "content": prompt},
|
||
]
|
||
|
||
# Apply chat template
|
||
text = tokenizer.apply_chat_template(
|
||
messages,
|
||
tokenize=False,
|
||
add_generation_prompt=True,
|
||
)
|
||
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||
|
||
# Generate
|
||
generated_ids = model.generate(
|
||
**model_inputs,
|
||
max_new_tokens=512,
|
||
temperature=0.7,
|
||
top_p=0.9,
|
||
)
|
||
|
||
# Decode output (removing the prompt tokens)
|
||
generated_ids = generated_ids[:, model_inputs.input_ids.shape[1]:]
|
||
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
||
print(response)
|
||
```
|
||
|
||
2) vLLM (Recommended for Production)
|
||
Karnak is compatible with vLLM for high-throughput inference.
|
||
|
||
Installation:
|
||
|
||
```bash
|
||
|
||
pip install -U vllm
|
||
```
|
||
|
||
Offline Inference:
|
||
|
||
```python
|
||
|
||
from vllm import LLM, SamplingParams
|
||
|
||
model_id = "Applied-Innovation-Center/Karnak"
|
||
|
||
# Initialize the model
|
||
llm = LLM(
|
||
model=model_id,
|
||
trust_remote_code=True,
|
||
max_model_len=20000, # Safe context range
|
||
tensor_parallel_size=1, # Adjust based on available GPUs
|
||
)
|
||
|
||
# Set sampling parameters
|
||
sampling_params = SamplingParams(
|
||
temperature=0.7,
|
||
top_p=0.9,
|
||
max_tokens=512,
|
||
)
|
||
|
||
# Generate
|
||
prompts = ["ما هي عاصمة مصر؟"]
|
||
outputs = llm.generate(prompts, sampling_params)
|
||
|
||
for o in outputs:
|
||
print(f"Prompt: {o.prompt}")
|
||
print(f"Generated: {o.outputs[0].text}")
|
||
```
|
||
|
||
Server Mode (OpenAI-Compatible API):
|
||
|
||
You can serve the model as an API compatible with OpenAI clients:
|
||
|
||
```bash
|
||
|
||
vllm serve "Applied-Innovation-Center/Karnak" \
|
||
--trust-remote-code \
|
||
--dtype bfloat16 \
|
||
--port 8000
|
||
```
|
||
|
||
Citation
|
||
If you use this model in your research or application, please cite it as follows:
|
||
|
||
```bibtex
|
||
@misc{karnak-40b,
|
||
title={Karnak: A Depth-Extended Arabic-English LLM},
|
||
year={2026},
|
||
publisher={Applied Innovation Center},
|
||
howpublished={\url{[https://huggingface.co/Applied-Innovation-Center/Karnak](https://huggingface.co/Applied-Innovation-Center/Karnak)}}
|
||
}
|
||
``` |