EduMentor-Qwen3-4B-v2-GGUF/README.md

---
license: apache-2.0
language:
- en
library_name: llama.cpp
pipeline_tag: text-generation
base_model: PraneetNS/EduMentor-Qwen3-4B-v2-FP16
tags:
- gguf
- llama.cpp
- qwen3
- engineering
- education
- mentor
- conversational
- ai
---

# EduMentor Qwen3 4B v2 (GGUF)

This repository contains GGUF quantized versions of **EduMentor Qwen3 4B v2**, an AI engineering mentor designed for university students, technical learning, coding assistance, project guidance, and placement preparation.

These models are converted from the merged FP16 checkpoint and optimized for inference with **llama.cpp**, **LM Studio**, **Ollama (GGUF import)**, **Open WebUI**, **Jan**, and other GGUF-compatible runtimes.

---

# Base Model

**FP16 Repository**

https://huggingface.co/PraneetNS/EduMentor-Qwen3-4B-v2-FP16

---

# Available Quantizations

| File | Size | Recommended For |
|------|------:|-----------------|
| EduMentor-Qwen3-4B-v2-F16.gguf | ~7.5 GB | Maximum quality |
| EduMentor-Qwen3-4B-v2-Q8_0.gguf | ~4.0 GB | Near-FP16 quality |
| EduMentor-Qwen3-4B-v2-Q6_K.gguf | ~3.1 GB | Best balance of quality and speed |
| EduMentor-Qwen3-4B-v2-Q4_K_M.gguf | ~2.4 GB | Fast local inference |

---

# Features

EduMentor v2 is fine-tuned for engineering education and technical mentoring.

It supports:

- Programming Fundamentals
- Data Structures & Algorithms
- System Design
- Machine Learning
- Deep Learning
- Large Language Models
- Electronics
- Electrical Engineering
- Mechanical Engineering
- Civil Engineering
- Mathematics
- Resume Reviews
- Internship Preparation
- Placement Preparation
- Interview Guidance
- Project Mentoring

---

# Response Format

EduMentor is optimized for multimodal tutoring systems.

Typical responses follow the structure:

```json
{
  "speech": "...",
  "display": {
    "type": "code | notes | roadmap | table | diagram",
    "content": "..."
  },
  "follow_up": "..."
}
```

This enables voice assistants to speak only natural explanations while rendering code, diagrams, or notes separately.

---

# Example

### User

Explain Binary Search.

### Assistant

```json
{
  "speech": "Binary Search repeatedly divides the search interval in half, making it much faster than linear search on sorted arrays.",

  "display": {
    "type": "code",
    "language": "python",
    "content": "def binary_search(...): ..."
  },

  "follow_up": "Would you like to understand its time complexity?"
}
```

---

# Running with llama.cpp

```bash
./llama-cli \
-m EduMentor-Qwen3-4B-v2-Q4_K_M.gguf \
-c 4096 \
-p "Explain recursion simply."
```

---

# Recommended Quantization

| Hardware | Recommendation |
|-----------|---------------|
| 8 GB RAM | Q4_K_M |
| 16 GB RAM | Q6_K |
| 24 GB+ RAM | Q8_0 |
| GPU Servers | F16 |

---

# Limitations

EduMentor may occasionally:

- produce incorrect technical information
- require verification for safety-critical engineering tasks
- generate imperfect JSON formatting for complex prompts
- benefit from retrieval augmentation for rapidly changing topics

The model should not replace professional engineering advice.

---

# Training

Base Architecture:

- Qwen3-4B

Fine-tuning:

- Supervised Fine-Tuning (LoRA)
- Fully merged into FP16
- Converted to GGUF using llama.cpp

---

# Creator

**Praneet N S**

EduMentor is an ongoing effort to build an AI mentor capable of providing high-quality engineering education through natural conversations and voice-first tutoring systems.