100 lines
3.0 KiB
Markdown
100 lines
3.0 KiB
Markdown
---
|
||
library_name: transformers
|
||
tags:
|
||
- reasoning
|
||
- sft
|
||
- unsloth
|
||
license: apache-2.0
|
||
language:
|
||
- en
|
||
pipeline_tag: text-generation
|
||
datasets:
|
||
- musaoc/Quran-reasoning-SFT
|
||
base_model:
|
||
- unsloth/Qwen3-0.6B
|
||
---
|
||
|
||
# Model Card for Quran-R1
|
||
|
||
|
||
## Model Details
|
||
|
||
This model is a fine-tuned version of Qwen/Qwen3-0.6B on the musaoc/Quran-reasoning-SFT dataset.
|
||
It is designed to perform reasoning and question-answering tasks related to the Quran, providing structured reasoning steps along with the final answer.
|
||
|
||
### Model Description
|
||
|
||
- **Language(s) (NLP):** English
|
||
- **License:** MIT
|
||
- **Fine-tuning method**: Supervised fine-tuning (SFT)
|
||
- **Finetuned from model:** Qwen3-0.6B
|
||
- **Dataset:** musaoc/Quran-reasoning-SFT
|
||
|
||
|
||
## Uses
|
||
|
||
The model is intended for:
|
||
|
||
- Educational purposes: Assisting with structured reasoning about Quranic content.
|
||
- Research: Exploring reasoning capabilities of small LLMs fine-tuned on religious text.
|
||
- QA Systems: Providing answers with reasoning traces.
|
||
|
||
Not intended for:
|
||
|
||
- Authoritative religious rulings (fatwas)
|
||
- Sensitive or controversial theological debates
|
||
- High-stakes decision making
|
||
|
||
### Out-of-Scope Use
|
||
|
||
- Scope: The model is limited to the reasoning dataset it was trained on. It may not generalize to broader Quranic studies.
|
||
|
||
## Bias, Risks, and Limitations
|
||
|
||
- Bias: Outputs reflect dataset biases and may not represent all scholarly interpretations.
|
||
- Hallucination risk: Like all LLMs, it may generate incorrect or fabricated reasoning.
|
||
- Religious sensitivity: Responses may not align with every sect, school, or interpretation. Use with caution in sensitive contexts.
|
||
|
||
## How to Get Started with the Model
|
||
|
||
Use the code below to get started with the model.
|
||
|
||
```python
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
||
tokenizer = AutoTokenizer.from_pretrained("khazarai/Quran-R1")
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
"khazarai/Quran-R1",
|
||
device_map={"": 0}
|
||
)
|
||
|
||
question = "How does the Quran address the issue of parental authority and children’s rights?"
|
||
|
||
messages = [
|
||
{"role" : "user", "content" : question}
|
||
]
|
||
text = tokenizer.apply_chat_template(
|
||
messages,
|
||
tokenize = False,
|
||
add_generation_prompt = True,
|
||
enable_thinking = True,
|
||
)
|
||
|
||
from transformers import TextStreamer
|
||
_ = model.generate(
|
||
**tokenizer(text, return_tensors = "pt").to("cuda"),
|
||
max_new_tokens = 512,
|
||
temperature = 0.6,
|
||
top_p = 0.95,
|
||
top_k = 20,
|
||
streamer = TextStreamer(tokenizer, skip_prompt = True)
|
||
)
|
||
```
|
||
|
||
|
||
## Training Data
|
||
|
||
**Dataset**: musaoc/Quran-reasoning-SFT
|
||
|
||
The Quranic Reasoning Question Answering (QRQA) Dataset is a synthetic dataset designed for experimenting purposes and for training and evaluating models capable of answering complex, knowledge-intensive questions about the Quran with a strong emphasis on reasoning.
|
||
This dataset is particularly well-suited for Supervised Fine-Tuning (SFT) of Large Language Models (LLMs) to enhance their understanding of Islamic scripture and their ability to provide thoughtful, reasoned responses. |