60 lines
2.5 KiB
Markdown
60 lines
2.5 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen2.5-0.5B-Instruct
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- unstuck
|
||
- task-breakdown
|
||
- adhd
|
||
- lora
|
||
- build-small-hackathon
|
||
- small-models
|
||
language:
|
||
- en
|
||
---
|
||
|
||
# Unstuck — Qwen2.5-0.5B fine-tuned for ADHD task breakdowns
|
||
|
||
A tiny (0.5B) instruct model fine-tuned to turn one overwhelming task into **tiny, timed, categorised steps** in the exact JSON schema used by [Unstuck](https://huggingface.co/spaces/build-small-hackathon/unstuck) (HF Build Small Hackathon, Backyard AI track).
|
||
|
||
It powers Unstuck's `UNSTUCK_BACKEND=finetuned` path — a fully local, serverless-free option that runs the same `generate(prompt) -> str` seam as the app's other backends.
|
||
|
||
## What it does
|
||
|
||
Given an overwhelming task, it returns only:
|
||
|
||
```json
|
||
{"steps":[{"text":"Open a trash bag and collect visible rubbish","category":"admin","est_minutes":5}, ...]}
|
||
```
|
||
|
||
- each step a single concrete action starting with an imperative verb,
|
||
- one category from `admin · creative · errand · deep-work`,
|
||
- a positive-integer minute estimate, **never above 25** (a hard app rule).
|
||
|
||
## How it was built
|
||
|
||
- **Base:** `Qwen/Qwen2.5-0.5B-Instruct`.
|
||
- **Data:** 130 schema-valid breakdowns **distilled** from a strong serverless model (`Qwen/Qwen3-30B-A3B-Instruct-2507` on Nebius Token Factory), each filtered through Unstuck's own validator so only on-contract examples survive. Purely synthetic — no user data.
|
||
- **Method:** LoRA (r=16, α=32, dropout 0.05; q/k/v/o projections), 3 epochs, lr 2e-4, merged into the base. Final train loss ≈ 0.21.
|
||
- **Compute:** a single **Modal** A10G GPU. Full pipeline (distill → train → verify): [`scripts/finetune/`](https://github.com/art87able/unstuck/tree/main/scripts/finetune).
|
||
|
||
## Usage
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
tok = AutoTokenizer.from_pretrained("art87able/unstuck-qwen2.5-0.5b-steps")
|
||
model = AutoModelForCausalLM.from_pretrained("art87able/unstuck-qwen2.5-0.5b-steps")
|
||
# Feed it Unstuck's breakdown prompt (see unstuck.prompts.breakdown_prompt) and parse the JSON.
|
||
```
|
||
|
||
Or in the app: `UNSTUCK_BACKEND=finetuned python app.py`.
|
||
|
||
## Limitations
|
||
|
||
A 0.5B model specialised for one narrow JSON task — it is not a general chat model. Outputs should still be schema-validated (the app does one repair retry on failure). Estimates are starting points; Unstuck recalibrates them to *your* real timings client-side.
|
||
|
||
- **Source:** https://github.com/art87able/unstuck
|
||
- **Live app:** https://huggingface.co/spaces/build-small-hackathon/unstuck
|