100 lines
3.0 KiB
Markdown
100 lines
3.0 KiB
Markdown
|
---
|
|||
|
|
library_name: transformers
|
|||
|
|
tags:
|
|||
|
|
- reasoning
|
|||
|
|
- sft
|
|||
|
|
- unsloth
|
|||
|
|
license: apache-2.0
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
datasets:
|
|||
|
|
- musaoc/Quran-reasoning-SFT
|
|||
|
|
base_model:
|
|||
|
|
- unsloth/Qwen3-0.6B
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Model Card for Quran-R1
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Model Details
|
|||
|
|
|
|||
|
|
This model is a fine-tuned version of Qwen/Qwen3-0.6B on the musaoc/Quran-reasoning-SFT dataset.
|
|||
|
|
It is designed to perform reasoning and question-answering tasks related to the Quran, providing structured reasoning steps along with the final answer.
|
|||
|
|
|
|||
|
|
### Model Description
|
|||
|
|
|
|||
|
|
- **Language(s) (NLP):** English
|
|||
|
|
- **License:** MIT
|
|||
|
|
- **Fine-tuning method**: Supervised fine-tuning (SFT)
|
|||
|
|
- **Finetuned from model:** Qwen3-0.6B
|
|||
|
|
- **Dataset:** musaoc/Quran-reasoning-SFT
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Uses
|
|||
|
|
|
|||
|
|
The model is intended for:
|
|||
|
|
|
|||
|
|
- Educational purposes: Assisting with structured reasoning about Quranic content.
|
|||
|
|
- Research: Exploring reasoning capabilities of small LLMs fine-tuned on religious text.
|
|||
|
|
- QA Systems: Providing answers with reasoning traces.
|
|||
|
|
|
|||
|
|
Not intended for:
|
|||
|
|
|
|||
|
|
- Authoritative religious rulings (fatwas)
|
|||
|
|
- Sensitive or controversial theological debates
|
|||
|
|
- High-stakes decision making
|
|||
|
|
|
|||
|
|
### Out-of-Scope Use
|
|||
|
|
|
|||
|
|
- Scope: The model is limited to the reasoning dataset it was trained on. It may not generalize to broader Quranic studies.
|
|||
|
|
|
|||
|
|
## Bias, Risks, and Limitations
|
|||
|
|
|
|||
|
|
- Bias: Outputs reflect dataset biases and may not represent all scholarly interpretations.
|
|||
|
|
- Hallucination risk: Like all LLMs, it may generate incorrect or fabricated reasoning.
|
|||
|
|
- Religious sensitivity: Responses may not align with every sect, school, or interpretation. Use with caution in sensitive contexts.
|
|||
|
|
|
|||
|
|
## How to Get Started with the Model
|
|||
|
|
|
|||
|
|
Use the code below to get started with the model.
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|||
|
|
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained("khazarai/Quran-R1")
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|||
|
|
"khazarai/Quran-R1",
|
|||
|
|
device_map={"": 0}
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
question = "How does the Quran address the issue of parental authority and children’s rights?"
|
|||
|
|
|
|||
|
|
messages = [
|
|||
|
|
{"role" : "user", "content" : question}
|
|||
|
|
]
|
|||
|
|
text = tokenizer.apply_chat_template(
|
|||
|
|
messages,
|
|||
|
|
tokenize = False,
|
|||
|
|
add_generation_prompt = True,
|
|||
|
|
enable_thinking = True,
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
from transformers import TextStreamer
|
|||
|
|
_ = model.generate(
|
|||
|
|
**tokenizer(text, return_tensors = "pt").to("cuda"),
|
|||
|
|
max_new_tokens = 512,
|
|||
|
|
temperature = 0.6,
|
|||
|
|
top_p = 0.95,
|
|||
|
|
top_k = 20,
|
|||
|
|
streamer = TextStreamer(tokenizer, skip_prompt = True)
|
|||
|
|
)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Training Data
|
|||
|
|
|
|||
|
|
**Dataset**: musaoc/Quran-reasoning-SFT
|
|||
|
|
|
|||
|
|
The Quranic Reasoning Question Answering (QRQA) Dataset is a synthetic dataset designed for experimenting purposes and for training and evaluating models capable of answering complex, knowledge-intensive questions about the Quran with a strong emphasis on reasoning.
|
|||
|
|
This dataset is particularly well-suited for Supervised Fine-Tuning (SFT) of Large Language Models (LLMs) to enhance their understanding of Islamic scripture and their ability to provide thoughtful, reasoned responses.
|