103 lines
4.5 KiB
Markdown
103 lines
4.5 KiB
Markdown
---
|
||
base_model:
|
||
- Qwen/Qwen2.5-Math-1.5B-Instruct
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# Lucida 1.5B
|
||
|
||
**Lucida** is a fine-tuned version of [Qwen2.5-Math-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B-Instruct) trained to decompose mathematical equations into hierarchical explanation trees that build genuine intuition.
|
||
|
||
Given any named equation, Lucida breaks it down into its meaningful sub-components — explaining not just *what* each part is, but *why* it exists and what changes when it grows or shrinks.
|
||
|
||
## What it does
|
||
|
||
Input: a LaTeX equation with an optional name and description.
|
||
|
||
Output: a structured decomposition tree where each node contains:
|
||
- The LaTeX fragment
|
||
- A node type (`expression`, `variable`, `constant`, `operator`, `function`, `other`)
|
||
- A short label (2–5 words)
|
||
- An intuition — the "aha" a great teacher would say
|
||
|
||
### Example
|
||
|
||
**Input:** `K = \frac{1}{2}mv^2` — Kinetic Energy
|
||
|
||
**Output:**
|
||
```
|
||
K = \frac{1}{2}mv^2 | expression | Kinetic Energy Equation | Moving objects store energy proportional to mass and the square of speed — doubling speed quadruples energy
|
||
K | variable | Kinetic Energy | Total mechanical energy of motion — zero when still, grows rapidly as speed increases
|
||
\frac{1}{2}mv^2 | expression | Energy of Motion | Mass times squared speed, halved — the ½ comes from integrating F=ma over distance from rest
|
||
\frac{1}{2} | other | Scaling Factor
|
||
m | variable | Mass | How much matter is moving — more mass means proportionally more energy at the same speed
|
||
v^2 | expression | Squared Speed | Speed multiplied by itself — squaring means fast objects carry disproportionately more energy than slow ones
|
||
v | variable | Speed | How fast the object is moving — the dominant factor since it appears squared
|
||
= | operator | —
|
||
```
|
||
|
||
## Training
|
||
|
||
Lucida was trained in two stages:
|
||
|
||
1. **SFT** on ~950 equations annotated in the 4-field compact format using frontier model annotations (Gemini 2.5 Flash) guided by 12 hand-written oracle examples spanning physics, chemistry, ML, calculus, linear algebra, economics, and probability.
|
||
|
||
2. **GRPO** starting from the SFT checkpoint, with three reward signals:
|
||
- **Format reward** — output parses cleanly into the compact tree format
|
||
- **Reconstruction reward** — children tokens cover their parent's tokens at every level
|
||
- **Judge reward** — LLM judge scores label quality, intuition quality, and structure quality against oracle exemplars
|
||
|
||
**Eval results (50 equations):**
|
||
|
||
| Metric | SFT baseline | Lucida (GRPO) |
|
||
|---|---|---|
|
||
| Parseable | 88% | 94% |
|
||
| Recon mean | 0.814 | 0.863 |
|
||
| Depth mean | 0.790 | 0.902 |
|
||
| Combined | 0.832 | 0.895 |
|
||
|
||
## Usage
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model_id = "rishiu/lucida-1.5b"
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")
|
||
|
||
SYSTEM_PROMPT = """You decompose mathematical equations into hierarchical explanation trees that teach intuition.
|
||
|
||
Output format — one line per node, 2 spaces of indent per depth level:
|
||
<latex_fragment> | <type> | <short_label> | <intuition>
|
||
|
||
Types: expression, variable, constant, operator, function, other
|
||
|
||
Rules:
|
||
- The root node is the full equation
|
||
- Recurse until every leaf is a single variable, named constant, or operator
|
||
- Short label: 2–5 words
|
||
- Intuition: the "aha" moment — for variables, what changes if this gets bigger?
|
||
- Omit intuition for operators and bare numeric factors"""
|
||
|
||
def decompose(latex, name=""):
|
||
user = f"Name: {name}\n" if name else ""
|
||
user += f"Equation: {latex}\n\nDecompose:"
|
||
messages = [
|
||
{"role": "system", "content": SYSTEM_PROMPT},
|
||
{"role": "user", "content": user},
|
||
]
|
||
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||
inputs = tokenizer(prompt, return_tensors="pt")
|
||
out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
|
||
generated = out[0][inputs["input_ids"].shape[1]:]
|
||
return tokenizer.decode(generated, skip_special_tokens=True).strip()
|
||
|
||
print(decompose(r"\frac{1}{2}mv^2", name="Kinetic Energy"))
|
||
```
|
||
|
||
## Limitations
|
||
|
||
- Best on standard named equations from physics, chemistry, ML, and mathematics
|
||
- May produce factual errors on highly specialized or obscure equations
|
||
- Partial derivative notation (e.g. ∂u/∂t) is occasionally split incorrectly into independent symbols
|
||
- Output quality depends on equation complexity — very long equations may be truncated |