7.5 KiB
Base vs fine-tuned on the same 10 test inputs
First 10 rows of the held-out test split (not cherry-picked). Both models get the identical prompt and identical output post-processing.
1. Tell me the ships in classs for kuha class
-- schema
CREATE TABLE table_name_52 (ships_in_class VARCHAR, class VARCHAR)
Gold
SELECT ships_in_class FROM table_name_52 WHERE class = "kuha class"
Base (raw output)
```sql
SELECT DISTINCT ships_in_class FROM table_name_52 WHERE class = 'kuha'
-> parsed: `SELECT DISTINCT ships_in_class FROM table_name_52 WHERE class = 'kuha'` | exact match: **no**
**Fine-tuned (raw output)**
SELECT ships_in_class FROM table_name_52 WHERE class = "kuha"
-> parsed: `SELECT ships_in_class FROM table_name_52 WHERE class = "kuha"` | exact match: **no**
---
### 2. What is the Drawn has an Against smaller than 20 and Points of 15?
```sql
-- schema
CREATE TABLE table_name_46 (drawn VARCHAR, against VARCHAR, points VARCHAR)
Gold
SELECT drawn FROM table_name_46 WHERE against < 20 AND points = 15
Base (raw output)
```sql
SELECT drawn FROM table_name_46 WHERE against < 20 AND points = '15';
-> parsed: `SELECT drawn FROM table_name_46 WHERE against < 20 AND points = '15'` | exact match: **no**
**Fine-tuned (raw output)**
SELECT drawn FROM table_name_46 WHERE against < 20 AND points = "15"
-> parsed: `SELECT drawn FROM table_name_46 WHERE against < 20 AND points = "15"` | exact match: **no**
---
### 3. How many picks for round 12?
```sql
-- schema
CREATE TABLE table_name_5 (pick__number VARCHAR, round VARCHAR)
Gold
SELECT COUNT(pick__number) FROM table_name_5 WHERE round = 12
Base (raw output)
SELECT COUNT(pick__number) FROM table_name_5 WHERE round = '12'
-> parsed: SELECT COUNT(pick__number) FROM table_name_5 WHERE round = '12' | exact match: no
Fine-tuned (raw output)
SELECT COUNT(pick__number) FROM table_name_5 WHERE round = "round 12"
-> parsed: SELECT COUNT(pick__number) FROM table_name_5 WHERE round = "round 12" | exact match: no
4. wich were the events when bout 2 was zhang ( chn ) l 0-5?
-- schema
CREATE TABLE table_19398910_4 (event VARCHAR, bout_2 VARCHAR)
Gold
SELECT event FROM table_19398910_4 WHERE bout_2 = "Zhang ( CHN ) L 0-5"
Base (raw output)
```sql
SELECT event FROM table_19398910_4 WHERE bout_2 = 'zhang (chn) l 0-5';
-> parsed: `SELECT event FROM table_19398910_4 WHERE bout_2 = 'zhang (chn) l 0-5'` | exact match: **YES**
**Fine-tuned (raw output)**
SELECT event FROM table_19398910_4 WHERE bout_2 = "Zhang ( CHN ) L 0-5"
-> parsed: `SELECT event FROM table_19398910_4 WHERE bout_2 = "Zhang ( CHN ) L 0-5"` | exact match: **YES**
---
### 5. Find the name and hours of project that has the most number of scientists.
```sql
-- schema
CREATE TABLE assignedto (project VARCHAR); CREATE TABLE projects (name VARCHAR, hours VARCHAR, code VARCHAR)
Gold
SELECT T1.name, T1.hours FROM projects AS T1 JOIN assignedto AS T2 ON T1.code = T2.project GROUP BY T2.project ORDER BY COUNT(*) DESC LIMIT 1
Base (raw output)
```sql
SELECT p.name, p.hours FROM projects AS p JOIN assignedto AS at ON p.code = at.code GROUP BY p.name, p.hours ORDER BY COUNT(*) DESC LIMIT 1;
-> parsed: `SELECT p.name, p.hours FROM projects AS p JOIN assignedto AS at ON p.code = at.code GROUP BY p.name, p.hours ORDER BY COUNT(*) DESC LIMIT 1` | exact match: **no**
**Fine-tuned (raw output)**
SELECT T1.name, T1.hours FROM projects AS T1 JOIN assignedto AS T2 ON T1.code = T2.project GROUP BY T1.name ORDER BY COUNT(*) DESC LIMIT 1
-> parsed: `SELECT T1.name, T1.hours FROM projects AS T1 JOIN assignedto AS T2 ON T1.code = T2.project GROUP BY T1.name ORDER BY COUNT(*) DESC LIMIT 1` | exact match: **no**
---
### 6. What is the total number of 3rd placed teams when the host is University of Manitoba, Winnipeg, Manitoba?
```sql
-- schema
CREATE TABLE table_1906920_1 (host VARCHAR)
Gold
SELECT COUNT(3 AS rd_place) FROM table_1906920_1 WHERE host = "University of Manitoba, Winnipeg, Manitoba"
Base (raw output)
SELECT COUNT(*) FROM table_1906920_1 WHERE host = 'University of Manitoba' AND position = 3
-> parsed: SELECT COUNT(*) FROM table_1906920_1 WHERE host = 'University of Manitoba' AND position = 3 | exact match: no
Fine-tuned (raw output)
SELECT COUNT(3 AS rd_placed) FROM table_1906920_1 WHERE host = "University of Manitoba, Winnipeg, Manitoba"
-> parsed: SELECT COUNT(3 AS rd_placed) FROM table_1906920_1 WHERE host = "University of Manitoba, Winnipeg, Manitoba" | exact match: no
7. What is the attendance for week 11?
-- schema
CREATE TABLE table_name_59 (attendance VARCHAR, week VARCHAR)
Gold
SELECT attendance FROM table_name_59 WHERE week = 11
Base (raw output)
SELECT attendance FROM table_name_59 WHERE week = '11'
-> parsed: SELECT attendance FROM table_name_59 WHERE week = '11' | exact match: no
Fine-tuned (raw output)
SELECT attendance FROM table_name_59 WHERE week = "11"
-> parsed: SELECT attendance FROM table_name_59 WHERE week = "11" | exact match: no
8. Name the syrian christians for kollam
-- schema
CREATE TABLE table_23214055_2 (syrian_christians VARCHAR, district VARCHAR)
Gold
SELECT syrian_christians FROM table_23214055_2 WHERE district = "Kollam"
Base (raw output)
```sql
SELECT syrian_christians FROM table_23214055_2 WHERE district = 'kollam'
-> parsed: `SELECT syrian_christians FROM table_23214055_2 WHERE district = 'kollam'` | exact match: **YES**
**Fine-tuned (raw output)**
SELECT syrian_christians FROM table_23214055_2 WHERE district = "Kollam"
-> parsed: `SELECT syrian_christians FROM table_23214055_2 WHERE district = "Kollam"` | exact match: **YES**
---
### 9. What is every entry for Saturday August 27 when the entry for Thursday August 25 is 23' 56.90 94.528mph?
```sql
-- schema
CREATE TABLE table_30058355_7 (sat_27_aug VARCHAR, thurs_25_aug VARCHAR)
Gold
SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = "23' 56.90 94.528mph"
Base (raw output)
```sql
SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = '23\' 56.90 94.528mph';
-> parsed: `SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = '23\' 56.90 94.528mph'` | exact match: **no**
**Fine-tuned (raw output)**
SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = "23' 56.90 94.528mph"
-> parsed: `SELECT sat_27_aug FROM table_30058355_7 WHERE thurs_25_aug = "23' 56.90 94.528mph"` | exact match: **YES**
---
### 10. Which Owner has a Name / Format of 105.3 kool fm - hot adult contemporary?
```sql
-- schema
CREATE TABLE table_name_18 (owner VARCHAR, name___format VARCHAR)
Gold
SELECT owner FROM table_name_18 WHERE name___format = "105.3 kool fm - hot adult contemporary"
Base (raw output)
```sql
SELECT owner FROM table_name_18 WHERE name___format = '105.3 kool fm - hot adult contemporary';
-> parsed: `SELECT owner FROM table_name_18 WHERE name___format = '105.3 kool fm - hot adult contemporary'` | exact match: **YES**
**Fine-tuned (raw output)**
SELECT owner FROM table_name_18 WHERE name___format = "105.3 kool fm - hot adult contemporary"
-> parsed: `SELECT owner FROM table_name_18 WHERE name___format = "105.3 kool fm - hot adult contemporary"` | exact match: **YES**
---