---
license: apache-2.0
base_model: HuggingFaceTB/SmolLM2-135M-Instruct
datasets:
  - RomanTeucher/text2cypher-curated
language:
  - en
library_name: transformers
pipeline_tag: text-generation
tags:
  - text2cypher
  - cypher
  - neo4j
  - lora
---

# SmolLM2-135M-text2cypher

A fine-tune of [`HuggingFaceTB/SmolLM2-135M-Instruct`](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct)
that turns a natural-language question + a graph schema into a **Cypher** query.
Trained with LoRA on [`RomanTeucher/text2cypher-curated`](https://huggingface.co/datasets/RomanTeucher/text2cypher-curated)
(1000 train / 75 val / 50 test); the adapters are **merged into the base weights**,
so this is a standalone full model that loads like any HF checkpoint — no PEFT needed.
**This model is a submission for an interview.** 

## Usage

The model expects the same ChatML prompt it was trained on: a fixed system message
plus a user turn of `Schema:` + `Question:`.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "rizwan261/smollm2-135m-text2cypher"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

SYSTEM = ("You are a text-to-Cypher engine for Neo4j. Given a graph schema and a "
          "question, output a single valid Cypher query that answers the question. "
          "Use only labels, relationship types and properties that appear in the "
          "schema. Output the Cypher query and nothing else: no explanation, no "
          "comments, no markdown code fences.")

schema = "Graph schema: Relevant node labels and their properties (with datatypes) are:\nUpdateDate {update_date: DATE}"
question = "Which nodes are connected to UpdateDate where update_date is 2008-01-29, and also to another node?"
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion:\n{question}"},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

```

## Training

LoRA (r=16, α=32, dropout=0.05) on the attention + MLP projections, completion-only
loss (the prompt is masked, loss is on the target Cypher only), then merged.

| | |
|---|---|
| epochs | 20 |
| learning rate | 5e-5, cosine, 5% warmup |
| effective batch | 16 (bs 4 × grad-accum 4) |
| max length | 1024 |
| best val loss | ~0.32 (by `eval_loss`) |

## Evaluation

Test split (n=50), greedy decoding. **Translation / structural metrics** over the
whole split — surface proxies for correctness:

| metric | base | this model |
| --- | ---: | ---: |
| exact_match | 0.00 | 0.26 |
| structural_f1 | 0.18 | 0.81 |
| google_bleu | 0.07 | 0.70 |
| well_formed | 0.00 | 0.98 |

**Execution-based** evaluation against the live Neo4j demo databases, on the 18/50
samples that carry a database reference — predicted and gold queries are run and
their *result sets* compared. This is the honest measure, and each stricter layer
peels back the previous one's optimism:

| how strict | metric | base | this model |
| --- | --- | ---: | ---: |
| real parser + live schema | CyVer KG-Valid-Query | 0.00 | 0.61 |
| runs on the DB | query executes | 0.00 | 0.72 |
| **returns the right rows** | exec ExactMatch (non-trivial) | 0.00 | **0.00** |

The surface metrics jump, but execution accuracy stays at zero: the model writes
well-formed, schema-valid, runnable Cypher that returns the *wrong data*.

## Limitations

A 135M model produces Cypher that often *looks* right but isn't: only ~26% are
exact matches, and **non-trivial execution accuracy is ~0** — well-formed,
runnable queries that return the wrong rows. It reliably gets node labels, clauses
and the query skeleton, but struggles with same-node/co-reference patterns,
multi-stage aggregation, and occasionally hallucinates functions. Execution
numbers come from a small (n=18) subset run against live demo databases, so they
can drift.

## License

Apache-2.0, inherited from the SmolLM2 base model.