169 lines
3.4 KiB
Markdown
169 lines
3.4 KiB
Markdown
---
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
library_name: llama.cpp
|
|
pipeline_tag: text-generation
|
|
base_model: PraneetNS/EduMentor-Qwen3-4B-v2-FP16
|
|
tags:
|
|
- gguf
|
|
- llama.cpp
|
|
- qwen3
|
|
- engineering
|
|
- education
|
|
- mentor
|
|
- conversational
|
|
- ai
|
|
---
|
|
|
|
# EduMentor Qwen3 4B v2 (GGUF)
|
|
|
|
This repository contains GGUF quantized versions of **EduMentor Qwen3 4B v2**, an AI engineering mentor designed for university students, technical learning, coding assistance, project guidance, and placement preparation.
|
|
|
|
These models are converted from the merged FP16 checkpoint and optimized for inference with **llama.cpp**, **LM Studio**, **Ollama (GGUF import)**, **Open WebUI**, **Jan**, and other GGUF-compatible runtimes.
|
|
|
|
---
|
|
|
|
# Base Model
|
|
|
|
**FP16 Repository**
|
|
|
|
https://huggingface.co/PraneetNS/EduMentor-Qwen3-4B-v2-FP16
|
|
|
|
---
|
|
|
|
# Available Quantizations
|
|
|
|
| File | Size | Recommended For |
|
|
|------|------:|-----------------|
|
|
| EduMentor-Qwen3-4B-v2-F16.gguf | ~7.5 GB | Maximum quality |
|
|
| EduMentor-Qwen3-4B-v2-Q8_0.gguf | ~4.0 GB | Near-FP16 quality |
|
|
| EduMentor-Qwen3-4B-v2-Q6_K.gguf | ~3.1 GB | Best balance of quality and speed |
|
|
| EduMentor-Qwen3-4B-v2-Q4_K_M.gguf | ~2.4 GB | Fast local inference |
|
|
|
|
---
|
|
|
|
# Features
|
|
|
|
EduMentor v2 is fine-tuned for engineering education and technical mentoring.
|
|
|
|
It supports:
|
|
|
|
- Programming Fundamentals
|
|
- Data Structures & Algorithms
|
|
- System Design
|
|
- Machine Learning
|
|
- Deep Learning
|
|
- Large Language Models
|
|
- Electronics
|
|
- Electrical Engineering
|
|
- Mechanical Engineering
|
|
- Civil Engineering
|
|
- Mathematics
|
|
- Resume Reviews
|
|
- Internship Preparation
|
|
- Placement Preparation
|
|
- Interview Guidance
|
|
- Project Mentoring
|
|
|
|
---
|
|
|
|
# Response Format
|
|
|
|
EduMentor is optimized for multimodal tutoring systems.
|
|
|
|
Typical responses follow the structure:
|
|
|
|
```json
|
|
{
|
|
"speech": "...",
|
|
"display": {
|
|
"type": "code | notes | roadmap | table | diagram",
|
|
"content": "..."
|
|
},
|
|
"follow_up": "..."
|
|
}
|
|
```
|
|
|
|
This enables voice assistants to speak only natural explanations while rendering code, diagrams, or notes separately.
|
|
|
|
---
|
|
|
|
# Example
|
|
|
|
### User
|
|
|
|
Explain Binary Search.
|
|
|
|
### Assistant
|
|
|
|
```json
|
|
{
|
|
"speech": "Binary Search repeatedly divides the search interval in half, making it much faster than linear search on sorted arrays.",
|
|
|
|
"display": {
|
|
"type": "code",
|
|
"language": "python",
|
|
"content": "def binary_search(...): ..."
|
|
},
|
|
|
|
"follow_up": "Would you like to understand its time complexity?"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
# Running with llama.cpp
|
|
|
|
```bash
|
|
./llama-cli \
|
|
-m EduMentor-Qwen3-4B-v2-Q4_K_M.gguf \
|
|
-c 4096 \
|
|
-p "Explain recursion simply."
|
|
```
|
|
|
|
---
|
|
|
|
# Recommended Quantization
|
|
|
|
| Hardware | Recommendation |
|
|
|-----------|---------------|
|
|
| 8 GB RAM | Q4_K_M |
|
|
| 16 GB RAM | Q6_K |
|
|
| 24 GB+ RAM | Q8_0 |
|
|
| GPU Servers | F16 |
|
|
|
|
---
|
|
|
|
# Limitations
|
|
|
|
EduMentor may occasionally:
|
|
|
|
- produce incorrect technical information
|
|
- require verification for safety-critical engineering tasks
|
|
- generate imperfect JSON formatting for complex prompts
|
|
- benefit from retrieval augmentation for rapidly changing topics
|
|
|
|
The model should not replace professional engineering advice.
|
|
|
|
---
|
|
|
|
# Training
|
|
|
|
Base Architecture:
|
|
|
|
- Qwen3-4B
|
|
|
|
Fine-tuning:
|
|
|
|
- Supervised Fine-Tuning (LoRA)
|
|
- Fully merged into FP16
|
|
- Converted to GGUF using llama.cpp
|
|
|
|
---
|
|
|
|
# Creator
|
|
|
|
**Praneet N S**
|
|
|
|
EduMentor is an ongoing effort to build an AI mentor capable of providing high-quality engineering education through natural conversations and voice-first tutoring systems. |