qwen-english-mcq/README.md

---
license: mit
language:
- en
pipeline_tag: text-generation
tags:
- causal-lm
- unsloth
- dpo
- sft
- mcq-generator
- education
base_model: Qwen/Qwen3-1.7B
---

# Qwen English MCQ Generator (Instruction-less) 📝

An optimized, instruction-less causal language model fine-tuned for high-quality **Multiple Choice Question (MCQ) generation** from English documents. 

This model is based on `Qwen/Qwen3-1.7B` and has been aligned using **Supervised Fine-Tuning (SFT)** and **Direct Preference Optimization (DPO)** via Unsloth.

## Project Resources

- **GitHub Repository:** [https://github.com/Akil-x/Doc-to-MCQ-Generator](https://github.com/Akil-x/Doc-to-MCQ-Generator)

## Model Details

- **Developed by:** Akil Al-Sharafi, Jihad Nassari, Bashar Alwan, Adeeb Hassan, and Akram AL-subari
- **Base Model:** `Qwen/Qwen3-1.7B`
- **Training Method:** SFT (LoRA) + DPO (LoRA) via Unsloth
- **Inference Mode:** Causal Auto-Completion (Instruction-less / Prompt-less)
- **Primary Domain:** Educational Assessment / Automatic Exam Generation

## Accompanying Research Paper

While this English model is developed as a sister model using a similar SFT + DPO fine-tuning pipeline, the core methodology, optimization frameworks, and evaluations are detailed in the accompanying research paper (which focuses primarily on the Arabic model):

**Paper Title:** *Bridging the Performance Gap in Arabic MCQ Generation: Instruction-less Fine-Tuning of Compact 1.7B Models via DPO and RLAIF to Match Giant 120B LLMs*  
**Preprint / DOI:** [10.5281/zenodo.20533068](https://doi.org/10.5281/zenodo.20533068)  
**Zenodo Record:** [https://zenodo.org/records/20533068](https://zenodo.org/records/20533068)

```bibtex
@misc{alsharafi2026bridging,
  author       = {Al-Sharafi, Akil and Nassari, Jihad and Alwan, Bashar and Hassan, Adeeb and AL-subari, Akram},
  title        = {Bridging the Performance Gap in Arabic MCQ Generation: Instruction-less Fine-Tuning of Compact 1.7B Models via DPO and RLAIF to Match Giant 120B LLMs},
  year         = 2026,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.20533068},
  url          = {https://doi.org/10.5281/zenodo.20533068}
}
```

---

## How It Works (Instruction-less Causal Auto-Completion)

Unlike general-purpose models that require natural language instructions (e.g. *"Generate a question..."*), this model acts as a pure **causal text completer** trained on custom XML tags. It expects the text context wrapped in `<context>...</context>` followed by the opening `<question>` tag. It will naturally autocomplete the MCQ sequence (Question, Answer, and Distractors) and stop generating at `</distractors>`.

### Prompt Prefix Format
```xml
<context>
[Input text segment from document]
</context>
<question>
```

---

## Usage Example (Python)

You can load and run this model locally using standard Hugging Face `transformers`:

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "akilx/qwen-english-mcq"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.float16,
    load_in_4bit=True  # Recommended for low VRAM
)

# Format the input prefix
context_text = "Artificial intelligence (AI) is intelligence demonstrated by machines, as opposed to natural intelligence."
prompt = f"<context>\n{context_text}\n</context>\n<question>\n"

inputs = tokenizer([prompt], return_tensors="pt").to("cuda")

# Generate completion
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        use_cache=True,
        stop_strings=["</distractors>"],
        tokenizer=tokenizer,
        temperature=0.1
    )

# Decode output
decoded = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
generated_mcq = decoded.split("</context>")[-1].strip()
print(generated_mcq)
```

### Expected Generated Output
```xml
What is intelligence demonstrated by machines called?
</question>
<answer>
Artificial intelligence
</answer>
<distractors>
- Natural intelligence
- Machine learning
- Cognitive science
</distractors>
```