---
license: apache-2.0
language:
- en
base_model:
- HuggingFaceTB/SmolLM2-135M-Instruct
pipeline_tag: text-generation
library_name: transformers
tags:
- text2cypher
- cypher
- graph-databases
- supervised-fine-tuning
- cpu-training
---

# SmolLM2-135M Text2Cypher

Fine-tuned `HuggingFaceTB/SmolLM2-135M-Instruct` for generating Cypher queries from a graph schema and a natural-language question.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "oscardean/smollm2-135m-text2cypher"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

messages = [
    {
        "role": "system",
        "content": (
            "You translate natural-language questions into Cypher queries. "
            "Use only the supplied graph schema and return only the Cypher query."
        ),
    },
    {
        "role": "user",
        "content": (
            "Graph schema:\n"
            "Person {name: STRING}\n"
            "Movie {title: STRING, year: INTEGER}\n"
            "(Person)-[:DIRECTED]->(Movie)\n\n"
            "Question:\n"
            "Which movies did Christopher Nolan direct before 2010?"
        ),
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
)

outputs = model.generate(
    inputs,
    max_new_tokens=192,
    do_sample=False,
)

prediction = tokenizer.decode(
    outputs[0, inputs.shape[1]:],
    skip_special_tokens=True,
)

print(prediction)
```

## Training

| Hyperparameter          |                  Value |
| ----------------------- | ---------------------: |
| Training samples        |                  1,000 |
| Validation samples      |                     75 |
| Epochs                  |                      3 |
| Learning rate           |                 `5e-5` |
| Batch size              |                      2 |
| Gradient accumulation   |                      4 |
| Effective batch size    |                      8 |
| Weight decay            |                 `0.01` |
| Warmup ratio            |                 `0.05` |
| Maximum sequence length |                    800 |
| Decoding                |                 Greedy |
| Checkpoint selection    | Lowest validation loss |

## Evaluation

Evaluated on the 50-sample test split.

| Metric                 |   Base | Fine-tuned |
| ---------------------- | -----: | ---------: |
| Basic query structure  |  2.00% |    100.00% |
| Token F1               | 12.35% |     55.20% |
| Node-label agreement   |  0.00% |     58.00% |
| Component match rate   | 29.20% |     49.60% |
| Normalized exact match |  0.00% |      0.00% |

## Limitations

* May hallucinate labels, relationships, or properties.
* May omit filters, constants, or return fields.
* May repeat conditions.
* May use incorrect relationship directions or operators.
* May generate SQL-like syntax instead of valid Cypher.
* Can produce structurally plausible but semantically incorrect queries.
* Should be validated before execution.
* Not intended for direct production use.