Qwen2.5-3B-Egyptian-MCQ/README.md

# 🇪🇬 Egyptian Ministry English MCQ Generator

A fine-tuned **Qwen2.5-3B-Instruct** model specialized in generating English multiple-choice questions that closely follow the style of the **Egyptian Ministry of Education (General Secondary Stage)**.
The model was instruction-tuned using **LoRA (PEFT)** on a synthetic dataset designed to teach the model the structure, difficulty, and writing style of official Egyptian English examinations.

---

## Features
- Ministry-style English MCQs
- Grammar question generation
- Vocabulary question generation
- Topic-based generation
- Unit-based generation
- Curriculum-aligned outputs
- JSON-only responses
- One correct answer with three plausible distractors
- Explanation for every generated question
- Optimized for production deployment with **vLLM**

---
# Base Model

| Property | Value |
|----------|-------|
| Base Model | Qwen/Qwen2.5-3B-Instruct |
| Fine-tuning Method | LoRA (PEFT) |
| Training Framework | Unsloth + TRL |
| Inference | Hugging Face Transformers / vLLM |
| Precision | FP16 |

---

# Training Dataset
The model was trained on a synthetic instruction dataset specifically designed for Egyptian Ministry English examinations.

### Dataset Statistics
| Split | Samples |
|-------|---------:|
| Training | ~5,751 |
| Validation | ~640 |
| Total | ~6,400 |

The dataset contains instruction-response conversations covering:
- Grammar
- Vocabulary
- Ministry examination style
- Curriculum units
- Grammar focus
- Topic-based generation
- JSON formatting
Instead of memorizing questions, the model learns how official ministry questions are structured.
---

# Output Format

The model always returns structured JSON.
Example:
```json
{
  "questions": [
    {
      "statement": "...",
      "correct_answer": "...",
      "plausible_distractors": [
        "...",
        "...",
        "..."
      ],
      "explanation": "..."
    }
  ]
}
```

---

# Example Prompt
### System Prompt
```
You are a Senior English Assessment Specialist and an official item writer for the Egyptian Ministry of Education.
Generate authentic English examination questions.
Return ONLY valid JSON.
```
### User Prompt

```
Generate 10 grammar questions.

Grammar Focus:
Past Perfect / Past Perfect Continuous

Return only valid JSON.
```

---

# Supported Tasks
The model supports requests such as:
- Generate grammar questions
- Generate vocabulary questions
- Generate Unit 7 questions
- Generate questions about "At the Airport"
- Generate Ministry-style MCQs
- Generate questions for a specific grammar focus
- Generate curriculum-based assessments
---

# Training Configuration
| Parameter | Value |
|-----------|------:|
| Base Model | Qwen2.5-3B-Instruct |
| LoRA Rank | 16 |
| LoRA Alpha | 32 |
| LoRA Dropout | 0.05 |
| Learning Rate | 1e-4 |
| Epochs | 2 |
| Optimizer | AdamW 8-bit |
| Maximum Sequence Length | 2048 |

---

# How to Use
## Option 1 — Transformers

Install the required libraries:

```bash
pip install transformers torch
```

Load the model:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_ID = "tokhey/Qwen2.5-3B-Egyptian-MCQ"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype="auto",
    device_map="auto",
)
```

Prepare the conversation:

```python
messages = [
    {
        "role": "system",
        "content": SYSTEM_PROMPT
    },
    {
        "role": "user",
        "content": USER_PROMPT
    }
]
```

Apply the chat template:

```python
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
```

Generate:

```python
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1200,
    temperature=0.7,
    top_p=0.9,
)

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[1]:],
    skip_special_tokens=True,
)

print(response)
```

---

## Option 2 — vLLM (Recommended)

Start the server:

```bash
python -m vllm.entrypoints.openai.api_server \
    --model tokhey/Qwen2.5-3B-Egyptian-MCQ
```

Or use Python:

```python
from vllm import LLM
from vllm import SamplingParams

llm = LLM(
    model="tokhey/Qwen2.5-3B-Egyptian-MCQ"
)

sampling_params = SamplingParams(
    temperature=0.7,
    top_p=0.9,
    max_tokens=1200,
)
```

Generate:

```python
outputs = llm.generate(
    text ,
    sampling_params,
)

print(outputs[0].outputs[0].text)
```

---

# Intended Use

This model is intended for:

- Educational platforms
- Automatic exam generation
- AI teaching assistants
- English assessment generation
- Curriculum-aware educational systems

---

# Limitations

- Optimized specifically for Egyptian secondary education.
- Designed for English examination generation.
- Outputs should be reviewed before being used in official assessments.

---

---

# Tech Stack

- PyTorch
- Hugging Face Transformers
- PEFT
- TRL
- Unsloth
- vLLM

---

# License

This project follows the same license as the base model (**Qwen2.5-3B-Instruct**). Please refer to the base model repository for licensing details.