qwen2.5-3b-nl2sql-spider-grpo/README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-3B-Instruct
tags:
  - text-to-sql
  - nl2sql
  - spider
  - grpo
  - reinforcement-learning
  - tool-use
  - verl
  - oumi
language:
  - en
datasets:
  - spider
pipeline_tag: text-generation
---

# Qwen2.5-3B-Instruct — NL2SQL Spider GRPO (step 100)

`Qwen/Qwen2.5-3B-Instruct` trained with **GRPO** in a tool-calling environment where the
model issues real `run_sql` queries against live SQLite databases and is rewarded on whether
its final query returns the same rows as the gold query.

Trained with [verl](https://github.com/volcengine/verl) 0.7.1 through
[Oumi](https://github.com/oumi-ai/oumi)'s `DatabaseExecutableEnvironment`.

## Results — Spider dev (1034 rows, greedy, single sample)

| metric | base | **this model** | change |
|---|---|---|---|
| **official test-suite execution accuracy** | 0.336 | **0.644** | **×1.92** |
| official exact-set-match | 0.195 | 0.397 | ×2.04 |
| legacy `evaluation.py` execution | 0.264 | 0.583 | ×2.21 |

**Quote the test-suite number (0.644).** It is the current official Spider metric
(Zhong, Yu & Klein, EMNLP 2020, [`taoyds/test-suite-sql-eval`](https://github.com/taoyds/test-suite-sql-eval)).
The legacy `evaluation.py` under-reports this model by 6.1 points because 242 of its 1034
predictions cannot be parsed by Spider's `process_sql.py` (subqueries, trailing `;`, `AS`
aliases), and an unparseable prediction scores 0 regardless of the rows it returns.

### By difficulty (test-suite)

| | easy (248) | medium (446) | hard (174) | extra (166) | all |
|---|---|---|---|---|---|
| this model | 0.863 | 0.668 | 0.592 | 0.307 | **0.644** |
| base | 0.431 | 0.370 | 0.322 | 0.114 | 0.336 |

Multi-table reasoning is the bottleneck: easy 0.86 → extra-hard 0.31.

## Intended use

A research artifact demonstrating that **execution feedback works as an RL signal** for
text-to-SQL. It is not a state-of-the-art Spider model — published 3B systems reach 79–84%
using constrained decoding, schema ranking, full SFT, and multi-candidate inference, none of
which are used here. The result is 100 RL steps on a generic (non-code) instruct model.

## How it was trained

| | |
|---|---|
| algorithm | GRPO (`adv_estimator: grpo`, `norm_adv_by_std_in_grpo: true`), KL loss disabled |
| hardware | 8×H100, single node, ~80 s/step, ~2h30m total |
| batch | 64 prompts × n=8 = 512 trajectories/step |
| completion budget | 2048 tokens shared across all turns |
| lr | 1e-6 constant, AdamW |
| sampling | train temp 1.0; eval greedy (t=0, n=1) |
| reward | binary result-set match against the gold query |
| train data | 6,997 of 7,000 `train_spider.json` rows (3 dropped — their *gold* SQL does not execute) |
| eval data | 1,034 `dev.json` rows, none dropped |

The reward is outcome-only: it looks at the final SQL and nothing else — not turn count, not
whether a tool was called, not style.

## Prompt format

The model expects a system message containing the full DDL of every table in the target
database, a user question, and hermes-style tool calling with a single `run_sql(query)` tool.
It answers with a fenced ```sql block.

```
[system] You are a SQL assistant for a SQLite database with this schema:
         CREATE TABLE singer (...)
         You may call the tool run_sql(query) to execute a SQL query and inspect
         results. When you are done, give your final answer as a single SQL query
         inside a ```sql code block.
[user]   How many singers do we have?
```

```
[assistant] <tool_call>{"name":"run_sql","arguments":{"query":"SELECT COUNT(*) FROM singer;"}}</tool_call>
[tool]      {"columns": ["COUNT(*)"], "rows": [[6]]}
[assistant] ```sql
            SELECT COUNT(*) FROM singer;
            ```
```

**To reproduce the numbers above you must replay the training tool loop.** One-shot prompting
measures a different system and understates badly. Tool responses must be truncated to 256
characters, middle-elided, exactly as verl does during training:

```python
half = max_length // 2
text = text[:half] + "...(truncated)..." + text[-half:]
```

## Behavioral profile

| | base | this model |
|---|---|---|
| valid executable SQL | — | 94% |
| answers with a query it actually ran | 43.9% | **85.4%** |
| `run_sql` calls per row | 1.79 | 1.02 |
| predictions unparseable by Spider's parser | 62.6% | 24.0% |

Training made tool use *precise* rather than eliminating it, and made SQL markedly more
standard despite the reward never mentioning style. The dominant remaining failure is **197
rows (19.1%) that run a query, see real rows, and still answer wrong** — schema grounding on
multi-table questions, not tool use and not syntax.

## Limitations

- Single seed, no error bars, one run per configuration.
- Evaluated on Spider **dev**. The held-out test split was never touched.
- Greedy single-sample only; no self-consistency or multi-candidate reranking.
- A later checkpoint (step 250) scores higher on the internal metric (0.752 vs 0.727) but was
  never run through official eval. **This checkpoint (step 100) is the one every number above
  refers to.**

## Citation

Environment and reward implementation: [oumi-ai/oumi](https://github.com/oumi-ai/oumi)
PRs #2560 (verl tool adapter), #2561 (NL2SQL env + reward), #2562 (Spider demo).