230 lines
7.4 KiB
Markdown
230 lines
7.4 KiB
Markdown
---
|
||
language:
|
||
- en
|
||
- ar
|
||
base_model: Qwen/Qwen3-4B-Instruct-2507
|
||
tags:
|
||
- text-generation-inference
|
||
- transformers
|
||
- unsloth
|
||
- qwen3
|
||
- fitness
|
||
- arabic
|
||
- bilingual
|
||
- agent
|
||
- json-output
|
||
pipeline_tag: text-generation
|
||
license: apache-2.0
|
||
---
|
||
|
||
# Qwen3-4B-FitGPT-AR-EN-Instruct
|
||
|
||
<p align="center">
|
||
<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="180"/>
|
||
</p>
|
||
|
||
## Overview
|
||
|
||
**Qwen3-4B-FitGPT-AR-EN-Instruct** is a specialized bilingual fitness AI model, fine-tuned on top of a custom-merged Qwen3-4B foundation. It is designed to deliver **science-based, practical fitness and nutrition guidance** in both **Arabic and English**, while maintaining strict instruction-following capabilities including structured JSON output for agent-based systems.
|
||
|
||
This is the **full merged model** (LoRA adapters merged into 16-bit weights), ready for direct deployment without any additional adapter loading.
|
||
|
||
---
|
||
|
||
## Model Details
|
||
|
||
| Property | Value |
|
||
|----------|-------|
|
||
| **Developed by** | Mohamed Ramadan |
|
||
| **Model Type** | Causal Language Model (Custom Merged Base + Fine-tuned) |
|
||
| **Base Architecture** | Custom DARE-TIES Merge of `Qwen3-4B-Instruct-2507` + `Qwen3-4B` |
|
||
| **Model Format** | Full Weights — LoRA adapters merged into 16-bit base |
|
||
| **Languages** | Arabic 🇸🇦 & English 🇺🇸 |
|
||
| **Training Framework** | Unsloth + Hugging Face TRL |
|
||
| **License** | Apache 2.0 |
|
||
|
||
---
|
||
|
||
## Key Capabilities
|
||
|
||
### 🏋️ Bilingual Fitness Expert
|
||
Delivers detailed, science-backed advice on:
|
||
- Workout programming & periodization
|
||
- Macro/micro nutrition planning
|
||
- Exercise technique and form cues
|
||
- Recovery and injury prevention
|
||
|
||
### 🤖 Strict Agent / JSON Mode
|
||
The model is trained to follow formatting instructions precisely:
|
||
- Returns **only valid JSON** when asked — no markdown wrappers, no preamble
|
||
- Returns **only a number** when asked for a number
|
||
- Returns **only a list** when asked for a list
|
||
- Never adds unsolicited commentary
|
||
|
||
### 🌍 Arabic-Native Support
|
||
Unlike most fitness models that treat Arabic as an afterthought, this model was fine-tuned with a dedicated Arabic fitness corpus (`CIDAR` + `alpaca-gpt4-arabic` + custom data), enabling fluent, natural Arabic responses.
|
||
|
||
---
|
||
|
||
## Training Pipeline
|
||
|
||
The model was developed through a **3-stage engineering pipeline**:
|
||
|
||
```
|
||
Stage 1 — Foundation Merging
|
||
Qwen3-4B-Instruct-2507 (55–70%)
|
||
+
|
||
Qwen3-4B Base (30–45%)
|
||
─────────────────────────────
|
||
Method: DARE-TIES (layer-wise weights)
|
||
Result: Custom bilingual base
|
||
|
||
Stage 2 — Supervised Fine-Tuning
|
||
~7,000 curated samples
|
||
Curriculum-ordered (easy → hard)
|
||
LoRA: r=64, alpha=128
|
||
Framework: Unsloth + TRL (SFT)
|
||
|
||
Stage 3 — Weight Integration
|
||
LoRA adapters merged into 16-bit base
|
||
Result: Standalone deployment-ready model
|
||
```
|
||
|
||
### Training Dataset Composition (~7,000 samples)
|
||
|
||
| Source | Domain | Language |
|
||
|--------|--------|----------|
|
||
| chibbss/fitness-chat | Fitness Q&A | EN |
|
||
| onurSakar/GYM-Exercise | Exercise library | EN |
|
||
| its-myrto/fitness-QA | Fitness Q&A | EN |
|
||
| Varick/workout-routine | Workout programs | EN |
|
||
| hammam/fitness-qa | Fitness synthetic | EN |
|
||
| arbml/CIDAR | General instructions | AR |
|
||
| alpaca-gpt4-arabic | General conversation | AR |
|
||
| mlabonne/FineTome-100k | Complex instructions | EN |
|
||
| Custom Agent examples | JSON / strict format | EN + AR |
|
||
|
||
All samples passed quality filters (deduplication, min-length, response quality) and were **curriculum-sorted** from easiest to hardest before training.
|
||
|
||
---
|
||
|
||
## System Prompts
|
||
|
||
For best results, use one of these system prompts:
|
||
|
||
**Fitness Coach (English):**
|
||
```
|
||
You are Qwen3-4B-FitGPT-AR-EN-Instruct, an elite fitness coach and sports nutritionist. Give science-based, detailed, personalised advice on training, nutrition, exercise technique, and recovery. Be specific and practical.
|
||
```
|
||
|
||
**Fitness Coach (Arabic):**
|
||
```
|
||
أنت Qwen3-4B-FitGPT-AR-EN-Instruct، مدرب لياقة بدنية نخبة وأخصائي تغذية رياضية. تقدّم نصائح علمية دقيقة ومخصصة في التدريب والتغذية وأداء التمارين والتعافي. كن تفصيلياً وعملياً ومستنداً إلى أحدث الأبحاث العلمية.
|
||
```
|
||
|
||
**Agent / Strict JSON Mode:**
|
||
```
|
||
You are a precise AI assistant. Follow every instruction exactly. If asked for JSON output — ONLY valid JSON, no markdown, no explanation, no text before or after. If asked for a number, return only the number. Never add unsolicited commentary.
|
||
```
|
||
|
||
---
|
||
|
||
## How to Use
|
||
|
||
### Option A — Transformers (Local)
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
import torch
|
||
|
||
model_id = "Mohamed132411/Qwen3-4B-FitGPT-AR-EN-Instruct"
|
||
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_id,
|
||
torch_dtype=torch.bfloat16,
|
||
device_map="auto"
|
||
)
|
||
|
||
system = "You are an elite fitness coach. Give science-based, practical advice."
|
||
user = "Create a weekly muscle-building plan for a 25-year-old male, 80 kg, beginner."
|
||
|
||
messages = [
|
||
{"role": "system", "content": system},
|
||
{"role": "user", "content": user}
|
||
]
|
||
|
||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||
inputs = tokenizer([text], return_tensors="pt").to("cuda")
|
||
|
||
with torch.no_grad():
|
||
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
|
||
|
||
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
||
```
|
||
|
||
### Option B — Unsloth (Faster, 4-bit)
|
||
|
||
```python
|
||
from unsloth import FastLanguageModel
|
||
|
||
model, tokenizer = FastLanguageModel.from_pretrained(
|
||
"Mohamed132411/Qwen3-4B-FitGPT-AR-EN-Instruct",
|
||
max_seq_length=2048,
|
||
load_in_4bit=True,
|
||
)
|
||
FastLanguageModel.for_inference(model)
|
||
```
|
||
|
||
### Option C — Agent / JSON Output
|
||
|
||
```python
|
||
import json
|
||
|
||
AGENT_SYSTEM = (
|
||
"You are a precise AI assistant. Follow every instruction exactly. "
|
||
"If asked for JSON output — ONLY valid JSON, no markdown, no explanation. "
|
||
"Never add unsolicited commentary."
|
||
)
|
||
|
||
messages = [
|
||
{"role": "system", "content": AGENT_SYSTEM},
|
||
{"role": "user", "content": "Return ONLY JSON: {exercise, sets, reps} for barbell squats."}
|
||
]
|
||
|
||
# ... generate as above, then:
|
||
response = model_generate(messages, temperature=0.1) # low temp for JSON
|
||
data = json.loads(response) # ✅ clean, parseable JSON
|
||
```
|
||
|
||
---
|
||
|
||
## Example Outputs
|
||
|
||
**Arabic fitness plan:**
|
||
> سؤال: أنا مبتدئ عمري 25 وزني 85 كغ طولي 178. أريد خطة تدريبية أسبوعية.
|
||
>
|
||
> *الموديل يرد بخطة تفصيلية بالعربي مع التمارين والتكرارات والتغذية المناسبة.*
|
||
|
||
**Strict JSON:**
|
||
> Prompt: `Return ONLY JSON {exercise, sets, reps} for squats.`
|
||
>
|
||
> Output: `{"exercise": "Barbell Squat", "sets": 4, "reps": 8}`
|
||
|
||
**Number-only:**
|
||
> Prompt: `How many grams of protein per kg for a strength athlete? Return only the integer.`
|
||
>
|
||
> Output: `2`
|
||
|
||
---
|
||
|
||
## Related Repositories
|
||
|
||
| Repo | Description |
|
||
|------|-------------|
|
||
| 🔗 [Qwen3-4B-FitGPT-AR-EN-Instruct](https://huggingface.co/Mohamed132411/Qwen3-4B-FitGPT-AR-EN-Instruct) | This repo — full 16-bit model |
|
||
| ⚡ [Qwen3-4B-FitGPT-AR-EN-Instruct-GGUF](https://huggingface.co/Mohamed132411/Qwen3-4B-FitGPT-AR-EN-Instruct-GGUF) | Q4_K_M quantized — for Ollama & llama.cpp |
|
||
|
||
---
|
||
|
||
*Built with ❤️ by Mohamed Ramadan using Unsloth + Hugging Face* |