103 lines
4.5 KiB
Markdown
103 lines
4.5 KiB
Markdown
|
---
|
|||
|
|
base_model:
|
|||
|
|
- Qwen/Qwen2.5-Math-1.5B-Instruct
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Lucida 1.5B
|
|||
|
|
|
|||
|
|
**Lucida** is a fine-tuned version of [Qwen2.5-Math-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B-Instruct) trained to decompose mathematical equations into hierarchical explanation trees that build genuine intuition.
|
|||
|
|
|
|||
|
|
Given any named equation, Lucida breaks it down into its meaningful sub-components — explaining not just *what* each part is, but *why* it exists and what changes when it grows or shrinks.
|
|||
|
|
|
|||
|
|
## What it does
|
|||
|
|
|
|||
|
|
Input: a LaTeX equation with an optional name and description.
|
|||
|
|
|
|||
|
|
Output: a structured decomposition tree where each node contains:
|
|||
|
|
- The LaTeX fragment
|
|||
|
|
- A node type (`expression`, `variable`, `constant`, `operator`, `function`, `other`)
|
|||
|
|
- A short label (2–5 words)
|
|||
|
|
- An intuition — the "aha" a great teacher would say
|
|||
|
|
|
|||
|
|
### Example
|
|||
|
|
|
|||
|
|
**Input:** `K = \frac{1}{2}mv^2` — Kinetic Energy
|
|||
|
|
|
|||
|
|
**Output:**
|
|||
|
|
```
|
|||
|
|
K = \frac{1}{2}mv^2 | expression | Kinetic Energy Equation | Moving objects store energy proportional to mass and the square of speed — doubling speed quadruples energy
|
|||
|
|
K | variable | Kinetic Energy | Total mechanical energy of motion — zero when still, grows rapidly as speed increases
|
|||
|
|
\frac{1}{2}mv^2 | expression | Energy of Motion | Mass times squared speed, halved — the ½ comes from integrating F=ma over distance from rest
|
|||
|
|
\frac{1}{2} | other | Scaling Factor
|
|||
|
|
m | variable | Mass | How much matter is moving — more mass means proportionally more energy at the same speed
|
|||
|
|
v^2 | expression | Squared Speed | Speed multiplied by itself — squaring means fast objects carry disproportionately more energy than slow ones
|
|||
|
|
v | variable | Speed | How fast the object is moving — the dominant factor since it appears squared
|
|||
|
|
= | operator | —
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Training
|
|||
|
|
|
|||
|
|
Lucida was trained in two stages:
|
|||
|
|
|
|||
|
|
1. **SFT** on ~950 equations annotated in the 4-field compact format using frontier model annotations (Gemini 2.5 Flash) guided by 12 hand-written oracle examples spanning physics, chemistry, ML, calculus, linear algebra, economics, and probability.
|
|||
|
|
|
|||
|
|
2. **GRPO** starting from the SFT checkpoint, with three reward signals:
|
|||
|
|
- **Format reward** — output parses cleanly into the compact tree format
|
|||
|
|
- **Reconstruction reward** — children tokens cover their parent's tokens at every level
|
|||
|
|
- **Judge reward** — LLM judge scores label quality, intuition quality, and structure quality against oracle exemplars
|
|||
|
|
|
|||
|
|
**Eval results (50 equations):**
|
|||
|
|
|
|||
|
|
| Metric | SFT baseline | Lucida (GRPO) |
|
|||
|
|
|---|---|---|
|
|||
|
|
| Parseable | 88% | 94% |
|
|||
|
|
| Recon mean | 0.814 | 0.863 |
|
|||
|
|
| Depth mean | 0.790 | 0.902 |
|
|||
|
|
| Combined | 0.832 | 0.895 |
|
|||
|
|
|
|||
|
|
## Usage
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
|
|||
|
|
model_id = "rishiu/lucida-1.5b"
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")
|
|||
|
|
|
|||
|
|
SYSTEM_PROMPT = """You decompose mathematical equations into hierarchical explanation trees that teach intuition.
|
|||
|
|
|
|||
|
|
Output format — one line per node, 2 spaces of indent per depth level:
|
|||
|
|
<latex_fragment> | <type> | <short_label> | <intuition>
|
|||
|
|
|
|||
|
|
Types: expression, variable, constant, operator, function, other
|
|||
|
|
|
|||
|
|
Rules:
|
|||
|
|
- The root node is the full equation
|
|||
|
|
- Recurse until every leaf is a single variable, named constant, or operator
|
|||
|
|
- Short label: 2–5 words
|
|||
|
|
- Intuition: the "aha" moment — for variables, what changes if this gets bigger?
|
|||
|
|
- Omit intuition for operators and bare numeric factors"""
|
|||
|
|
|
|||
|
|
def decompose(latex, name=""):
|
|||
|
|
user = f"Name: {name}\n" if name else ""
|
|||
|
|
user += f"Equation: {latex}\n\nDecompose:"
|
|||
|
|
messages = [
|
|||
|
|
{"role": "system", "content": SYSTEM_PROMPT},
|
|||
|
|
{"role": "user", "content": user},
|
|||
|
|
]
|
|||
|
|
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|||
|
|
inputs = tokenizer(prompt, return_tensors="pt")
|
|||
|
|
out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
|
|||
|
|
generated = out[0][inputs["input_ids"].shape[1]:]
|
|||
|
|
return tokenizer.decode(generated, skip_special_tokens=True).strip()
|
|||
|
|
|
|||
|
|
print(decompose(r"\frac{1}{2}mv^2", name="Kinetic Energy"))
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Limitations
|
|||
|
|
|
|||
|
|
- Best on standard named equations from physics, chemistry, ML, and mathematics
|
|||
|
|
- May produce factual errors on highly specialized or obscure equations
|
|||
|
|
- Partial derivative notation (e.g. ∂u/∂t) is occasionally split incorrectly into independent symbols
|
|||
|
|
- Output quality depends on equation complexity — very long equations may be truncated
|