---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
base_model: Qwen/Qwen3-0.6B
tags:
- jq
- code-generation
- json
- lora
datasets:
- gauthierpiarrette/nl2jq
metrics:
- execution-accuracy
---

# nl2jq-qwen3-0.6b

*Part of the **nl2jq** project — [all artifacts](https://huggingface.co/collections/gauthierpiarrette/nl2jq-natural-language-to-jq-locally-6a5299203df3832415275223) · [code + CLI](https://github.com/gauthierpiarrette/nl2jq) · [live demo](https://huggingface.co/spaces/gauthierpiarrette/nl2jq) · [benchmark](https://huggingface.co/datasets/gauthierpiarrette/nl2jq-bench)*

A LoRA fine-tune of [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) that translates a
natural-language request plus a JSON sample into a [jq](https://jqlang.github.io/jq/)
program. It is the **practical baseline** companion to the from-scratch
[`nl2jq-40m`](https://huggingface.co/gauthierpiarrette/nl2jq-40m): larger, but its
pretrained language priors make it markedly more reliable at **copying novel field names**
from your JSON — which is what most real CLI use needs.

The LoRA adapter has been merged into the base weights, so it loads as an ordinary model.

**This is the recommended default backend of the `jqgen` CLI** — the best
latency/accuracy balance for local use (~1–3s per query on laptop CPU).

## Usage

```bash
cat data.json | jqgen --backend qwen "total spend per customer, paid orders only"
```

Or directly:
```python
import re
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("gauthierpiarrette/nl2jq-qwen3-0.6b")
model = AutoModelForCausalLM.from_pretrained("gauthierpiarrette/nl2jq-qwen3-0.6b")
msgs = [{"role": "system", "content": "You translate a natural-language request plus a "
         "sample of JSON into a single jq program. Output only the jq program, nothing else."},
        {"role": "user", "content": "Request: highest paid person\nJSON sample: [...]"}]
enc = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
ids = enc["input_ids"]  # newer transformers return a BatchEncoding here, not a tensor
out = tok.decode(model.generate(ids, max_new_tokens=128)[0][ids.shape[1]:],
                 skip_special_tokens=True)
# Qwen3 emits an (empty) reasoning block before the program — strip it:
print(re.sub(r"(?s)^\s*<think>.*?</think>\s*", "", out).strip())
```

## Results

Execution accuracy on the frozen 400-item
[`nl2jq-bench v1.0.0`](https://huggingface.co/datasets/gauthierpiarrette/nl2jq-bench)
(0% field overlap with training, novel domains, one-shot evaluation). **These weights are
the v6 fine-tune** — trained on data with per-example-unique field names and coverage of
`reduce`/`foreach`/`walk`/`paths`/`if` constructs; the superseded v5 row is kept for the
record:

| Model | frozen pass@1 | valid | T1 | T2 | T3 | T4 | T5-gen |
|---|---|---|---|---|---|---|---|
| **this model (v6)** | **0.40** | 0.73 | 0.65 | 0.52 | 0.36 | 0.28 | **0.20** |
| v5 fine-tune (superseded) | 0.24 | 0.67 | 0.45 | 0.38 | 0.21 | 0.12 | 0.02 |
| Qwen3-0.6B base (zero-shot) | 0.01 | 0.06 | 0.02 | 0.00 | 0.00 | 0.00 | 0.02 |
| [nl2jq-qwen3.5-2b](https://huggingface.co/gauthierpiarrette/nl2jq-qwen3.5-2b) (bigger sibling, CLI config) | 0.48 | 0.82 | 0.75 | 0.61 | 0.47 | 0.35 | 0.18 |
| Claude Opus 4.8 (zero-shot, context row) | 0.96 | 0.98 | 1.00 | 0.96 | 0.96 | 0.96 | 0.90 |

Three honest readings:

- **Fine-tuning does all the work at this scale**: the base model zero-shot scores 0.01
  (it emits pseudo-code, not jq); this LoRA takes it to 0.40 on held-out fields.
- **The v6 data fix mattered more than anything else**: same base, same LoRA recipe,
  +67% pass@1 and 10× T5 purely from making field-copying the only winning strategy in
  training.
- **This is the fast backend, not the most accurate one**: it answers in ~1–3s on laptop
  CPU; the 2B sibling is +8 points at ~3× the latency. Both remain far from the frontier's
  0.96 — that gap is disclosed, not hidden.

## Training

- **Base**: Qwen3-0.6B. **Method**: LoRA (r=32, α=64, all-linear targets), 1 epoch over
  150k execution-verified examples, cosine schedule, bf16, gradient checkpointing.
- **Data**: the v6 generation of the synthetic pipeline — per-example-unique field names
  (so copying from the input, not vocabulary recall, is the only winning strategy) plus
  generators for `reduce`/`foreach`/`walk`/`paths`/`if`-banding constructs. 100% synthetic
  and execution-verified — see the
  [nl2jq dataset](https://huggingface.co/datasets/gauthierpiarrette/nl2jq). No web text, no
  scraped code. Every program was run under jq 1.7.1 and kept only if it produced
  non-degenerate output.
- Frontier-model outputs are **never** used as training targets; all natural-language
  phrasings are template/grammar-generated by the author.

## Limitations

- jq 1.7.1 only; no streaming, modules, or user-defined functions.
- Ambiguous requests may yield a valid program that answers a different reading.
- **Inspect before trusting the output** — the `jqgen` CLI prints the program to stderr.
  jq runs locally with no network access, but an incorrect program can still mislead.
- Not a general assistant — it emits jq and nothing else.

## License

Apache-2.0 (inherits the base model's license). Fine-tuned on CC BY 4.0 synthetic data.

Code, benchmark tooling, and training pipeline: [github.com/gauthierpiarrette/nl2jq](https://github.com/gauthierpiarrette/nl2jq)