208 lines
7.6 KiB
Markdown
208 lines
7.6 KiB
Markdown
---
|
||
license: other
|
||
license_name: qwen-research
|
||
license_link: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE
|
||
base_model: Qwen/Qwen2.5-3B-Instruct
|
||
language:
|
||
- tr
|
||
- en
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- unsloth
|
||
- qwen2.5
|
||
- lora
|
||
- sft
|
||
- meal-parsing
|
||
- nutrition
|
||
- calorie-estimation
|
||
- turkish
|
||
- structured-output
|
||
- json
|
||
---
|
||
|
||
# 🥗 Astra Meal Parser (v1)
|
||
|
||
A fine-tuned **Qwen2.5-3B-Instruct** model that reads a free-text meal description in
|
||
**Turkish or English** and turns it into a clean, structured list of food items and their
|
||
amounts — ready to feed into a deterministic nutrition calculator.
|
||
|
||
The model **does not** estimate calories or macros itself. It only parses. This is a
|
||
deliberate design choice (see *Why parsing only?* below) that keeps nutrition accuracy
|
||
high and easy to maintain.
|
||
|
||
```
|
||
"2 yumurta, 100g tavuk göğsü ve 1 muz"
|
||
│
|
||
▼ (this model — parsing)
|
||
{"items": [
|
||
{"name": "Yumurta", "amount": "2 adet"},
|
||
{"name": "Tavuk Göğsü", "amount": "100g"},
|
||
{"name": "Muz", "amount": "1 adet"}
|
||
]}
|
||
│
|
||
▼ (nutrition table + calculator — not part of this model)
|
||
{ totalCalories, totalProtein, totalCarbs, totalFat, items[...] }
|
||
```
|
||
|
||
- **Developed by:** Turhan Göksu
|
||
- **Model type:** Causal LM adapter merged into base weights (Qwen2.5-3B)
|
||
- **Languages:** Turkish, English, and mixed/code-switched input
|
||
- **Finetuned from:** [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)
|
||
|
||
## Why parsing only?
|
||
|
||
An earlier version asked the model to output calories and macros directly. It plateaued at
|
||
~25% calorie error with a systematic overestimation bias: a language model cannot reliably
|
||
memorize accurate per-food nutrition values, especially for foods with high natural variance.
|
||
|
||
Splitting the problem fixed this. The model now does the one thing language models are good
|
||
at — understanding messy natural language — and a static nutrition table + a small
|
||
calculator handle the arithmetic deterministically. Result: calorie error dropped from ~25%
|
||
to ~3%, and any remaining error is fixable by editing the table, **without retraining**.
|
||
|
||
## Output format
|
||
|
||
The model is trained to return **only** a strict JSON object:
|
||
|
||
```json
|
||
{"items": [{"name": "string", "amount": "string"}]}
|
||
```
|
||
|
||
No prose, no markdown, no macros — just the JSON.
|
||
|
||
## System prompt
|
||
|
||
Use this exact system prompt for best results:
|
||
|
||
```
|
||
You are a meal parser. Extract every food item and its amount from the user's meal
|
||
description (Turkish or English). Return ONLY a strict JSON object of the form
|
||
{"items": [{"name": string, "amount": string}]}. No macros, no calories, no
|
||
conversational text, no markdown, only valid JSON.
|
||
```
|
||
|
||
## Uses
|
||
|
||
### Direct use
|
||
- Meal logging / calorie tracking apps where users type meals in natural language.
|
||
- Bilingual and code-switched input such as `"200g grilled chicken ve 1 kase pirinç"`.
|
||
- A drop-in front end for a deterministic nutrition pipeline.
|
||
|
||
### Out-of-scope use
|
||
- **Standalone nutrition estimation.** This model only extracts items and amounts; it does
|
||
not produce calories or macros on its own.
|
||
- **Medical or dietary prescriptions.** Output is informational, not medical advice.
|
||
- **Open-ended conversation.** The model is specialized for structured parsing and is not
|
||
intended as a general assistant.
|
||
|
||
## How to get started
|
||
|
||
> **Note on inference.** This is a custom merged model and is **not served by the free
|
||
> Hugging Face Serverless Inference API**. Run it locally with `transformers`, convert it to
|
||
> GGUF for on-device / `llama.cpp` use, or deploy a dedicated Inference Endpoint.
|
||
|
||
```python
|
||
import json
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model_id = "Turhan123/astra-meal-parser" # pin a version with revision="v1"
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id, revision="v1")
|
||
model = AutoModelForCausalLM.from_pretrained(model_id, revision="v1", device_map="auto")
|
||
|
||
SYSTEM = (
|
||
"You are a meal parser. Extract every food item and its amount from the user's "
|
||
"meal description (Turkish or English). Return ONLY a strict JSON object of the form "
|
||
'{"items": [{"name": string, "amount": string}]}. '
|
||
"No macros, no calories, no conversational text, no markdown, only valid JSON."
|
||
)
|
||
|
||
def parse(meal: str):
|
||
messages = [{"role": "system", "content": SYSTEM},
|
||
{"role": "user", "content": meal}]
|
||
ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
|
||
return_tensors="pt").to(model.device)
|
||
out = model.generate(ids, max_new_tokens=256, do_sample=False)
|
||
text = tokenizer.decode(out[0][ids.shape[-1]:], skip_special_tokens=True)
|
||
return json.loads(text)
|
||
|
||
print(parse("2 yumurta, 100g tavuk göğsü ve 1 muz"))
|
||
# {'items': [{'name': 'Yumurta', 'amount': '2 adet'}, ...]}
|
||
```
|
||
|
||
## Training
|
||
|
||
| | |
|
||
|---|---|
|
||
| Base model | `Qwen/Qwen2.5-3B-Instruct` (4-bit QLoRA via Unsloth) |
|
||
| Method | Supervised fine-tuning, LoRA (r=16, α=32) on q/k/v/o/gate/up/down |
|
||
| Data | 778 meal→items examples (Turkish / English / mixed); 739 train / 39 eval |
|
||
| Schedule | 3 epochs, 279 steps, lr 2e-4, batch 4 × grad-accum 2, linear decay |
|
||
| Optimizer | AdamW 8-bit, weight decay 0.01 |
|
||
| Hardware | Single NVIDIA T4 (~20 min) |
|
||
| Export | LoRA merged into 16-bit weights |
|
||
|
||
## Evaluation
|
||
|
||
Held-out set of **94 meal descriptions** (53 Turkish, 34 English, 7 mixed), with zero
|
||
overlap with the training data. Parsing metrics score the model output directly; nutrition
|
||
metrics reflect the **full pipeline** (this parser + nutrition table + calculator).
|
||
|
||
**Parsing**
|
||
|
||
| Metric | Value |
|
||
|---|---|
|
||
| Item Precision / Recall / F1 | 100% / 100% / 100% |
|
||
| Parse failures | 0 / 94 |
|
||
| Unresolved foods (table gaps) | 0 |
|
||
|
||
**Nutrition (full pipeline)**
|
||
|
||
| Metric | Value |
|
||
|---|---|
|
||
| Calorie MAPE | 3.1% |
|
||
| Within ±15% | 85 / 91 |
|
||
| Protein / Carbs / Fat MAE | 0.5 g / 1.5 g / 0.4 g |
|
||
|
||
**Calorie MAPE by language**
|
||
|
||
| Language | MAPE | n |
|
||
|---|---|---|
|
||
| Turkish | 3.3% | 51 |
|
||
| English | 2.7% | 33 |
|
||
| Mixed (TR/EN) | 3.4% | 7 |
|
||
|
||
## Bias, risks, and limitations
|
||
|
||
- **Parsing only.** Calorie/macro accuracy depends on the accompanying nutrition table and
|
||
calculator, which are not part of this repository.
|
||
- **Portion ambiguity.** Vague amounts (e.g. "1 bowl of rice") are resolved with default
|
||
serving sizes; the true amount may differ. This is the dominant source of residual error.
|
||
- **Table coverage.** Foods outside the nutrition table cannot be scored downstream;
|
||
long-tail coverage is the main lever for production accuracy and is addressed by expanding
|
||
the table, not by retraining.
|
||
- **JSON robustness.** Output is valid JSON in the large majority of cases, but consuming
|
||
applications should still guard against an occasional malformed response (e.g. retry once).
|
||
|
||
## Versioning
|
||
|
||
Versions are published as git tags on this repository. Pin a specific version in production
|
||
with `revision="v1"`. Future improvements are added as new tags (`v2`, `v3`, …) without
|
||
breaking pinned consumers.
|
||
|
||
## Technical references
|
||
|
||
- Qwen2.5 Technical Report — [arXiv:2412.15115](https://arxiv.org/abs/2412.15115)
|
||
- LoRA: Low-Rank Adaptation of Large Language Models — [arXiv:2106.09685](https://arxiv.org/abs/2106.09685)
|
||
- Unsloth — [github.com/unslothai/unsloth](https://github.com/unslothai/unsloth)
|
||
|
||
## Acknowledgements
|
||
|
||
Built on [Qwen2.5](https://huggingface.co/Qwen) by the Qwen team, and trained efficiently
|
||
with [Unsloth](https://github.com/unslothai/unsloth).
|
||
|
||
## License
|
||
|
||
Fine-tuned from `Qwen/Qwen2.5-3B-Instruct`; use is subject to the
|
||
[Qwen Research License](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE)
|
||
of the base model. |