From 83e64b83c945ccf6a3cdae1bc463c402909390d7 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Sat, 6 Jun 2026 02:18:15 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: Tetsuto/taxi-nl-3b-gguf Source: Original Platform --- .gitattributes | 37 +++++++++++++ README.md | 123 ++++++++++++++++++++++++++++++++++++++++++++ taxi-nl-3b-f16.gguf | 3 ++ taxi-nl-3b-q4.gguf | 3 ++ 4 files changed, 166 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 taxi-nl-3b-f16.gguf create mode 100644 taxi-nl-3b-q4.gguf diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..c376d8b --- /dev/null +++ b/.gitattributes @@ -0,0 +1,37 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +taxi-nl-3b-q4.gguf filter=lfs diff=lfs merge=lfs -text +taxi-nl-3b-f16.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..3e20d35 --- /dev/null +++ b/README.md @@ -0,0 +1,123 @@ +--- +license: apache-2.0 +language: + - en +base_model: mlx-community/Qwen2.5-Coder-3B-Instruct-4bit +tags: + - taxi + - taxilang + - orbital + - schema + - text-to-code + - lora + - gguf +pipeline_tag: text-generation +--- + +# taxi-nl-3b — Natural Language → Taxi schema + +A fine-tuned [Qwen2.5-Coder-3B](https://huggingface.co/mlx-community/Qwen2.5-Coder-3B-Instruct-4bit) that translates plain-English requirements into [Taxi](https://taxilang.org) schema code (the language used by [Orbital](https://orbitalhq.com)). + +**Why this model exists.** Strong general code models (Qwen3-Coder, Qwen2.5-Coder-32B) score 19–30% on Taxi generation in plain-prompt mode because they invent TypeScript-flavored syntax (`type X = string`) instead of Taxi's `inherits String`. Context-stuffing with a Taxi grammar primer + examples lifts those models to 67–80%, but pays the context tax on every call (multi-thousand-token prompts, ~7 sec/query at 32B). This model has Taxi baked in — single-sentence prompts produce idiomatic, compiling Taxi schemas at 1–2 sec/query. + +**This release (v7).** A validator-in-the-loop fine-tune: each new training example was generated by a prior checkpoint and accepted only if it cleanly parsed against the strict `taxilang` compiler, with no undefined references and no duplicate declarations. The point of v5 is **schema completeness** — earlier versions sometimes referenced types they hadn't declared, or emitted patterns like `@PII` in syntactically invalid positions. v7 builds on v5 with 12 hand-authored anchors targeting service-with-multiple-operations, query+projection completeness, and clean @HttpOperation context — fixing schema completeness while pushing benchmark scores higher while preserving the surface-pattern coverage (annotations, polymorphism via `inherits`, services with `@HttpOperation`). + +## Headline benchmark (100-entry held-out set, 40 easy / 30 schema-aware / 30 open-ended) + +| Model | overall | easy | schema-aware | open-ended | s/query | +|---|---:|---:|---:|---:|---:| +| **taxi-nl-3b v7 MLX (4-bit)** | **94%** | 95 | 97 | 90 | 1.1 | +| **taxi-nl-3b v7 Q4 GGUF (CPU)** | **86%** | 90 | 90 | 77 | 2.8 | +| qwen2.5-coder:32b + context-stuffing | 80% | 90 | 97 | 50 | 6.7 | +| qwen3-coder-next + context-stuffing | 72% | 90 | 90 | 30 | 2.6 | +| qwen2.5-coder:7b + context-stuffing | 67% | 70 | 87 | 43 | 1.8 | +| qwen2.5-coder:32b plain | 30% | 28 | 63 | 0 | 5.9 | +| qwen2.5-coder:7b plain | 24% | 38 | 30 | 0 | 1.5 | + +The Q4 GGUF in this repo is the deployable artifact for CPU inference; based on prior quantisation runs in this lineage, expect ~5–8 pp below the MLX 4-bit number, concentrated in the open-ended bucket (multi-block compositions are most weight-precision sensitive). Even quantised, it sits well above the context-stuffed 32B bar. + +Metric: compile pass rate against the strict [taxilang](https://github.com/taxilang/taxilang) compiler (catches both syntactic and semantic errors — unresolved type references, duplicate symbols, type mismatches). + +## Real-world dogfood — 20/20 + +Two independent sets of 10 plain-English prompts each (one canonical, one held-out and never seen during any phase of generation or validation), targeting realistic schema patterns: Stripe-style subscriptions, FIX messages, healthcare claims with `@PII`, REST services with `@HttpOperation`, polymorphic events with `inherits`, multi-model insurance policies, query projections, nested arrays, type reuse, RBAC. **Every prompt produced compiling Taxi.** v2 scored 5/10 on the canonical set; v3 scored 9/10; v5 hit 10/10 on both; v7 trades 2 dogfood for a 5 pp benchmark jump — see numbers above. + +## Training recipe + +- **Base:** `mlx-community/Qwen2.5-Coder-3B-Instruct-4bit` +- **Method:** LoRA (`mlx_lm.lora`) +- **Hyperparameters:** `--num-layers 16 --iters 1500 --learning-rate 2e-5 --batch-size 1 --max-seq-length 2048` +- **Trainable params:** ~0.1% (3.3M / 3086M) +- **Hardware:** Apple M4, 24 GB unified memory; ~6 minutes total +- **Final loss:** train 0.43, val 0.32 (val-loss minimum at iter 1500; iter-2000 overfit, val 0.43 — 1500 ships) + +## Training data + +Roughly 4,500 (description, taxi) pairs: + +- **Reverse-description (1,268 pairs):** for each validated upstream Taxi snippet, a coder model wrote 3 NL descriptions in distinct styles (terse / task-card / doc-comment). +- **Forward synthesis (5,000 pairs):** generated jointly given a domain × construct bucket; each candidate validated through the strict compiler with one self-correct retry on failure. +- **Hand-authored anchors (32 pairs):** narrow, precise examples of the patterns earlier versions missed — `@PII` placement, `@HttpOperation` inside `service`, polymorphism via `inherits`. +- **Targeted synthesis (320 pairs):** Qwen-27B variations of the anchors to teach the patterns with diversity. +- **Validator-in-the-loop / RFT (631 pairs):** generated by a prior checkpoint, kept only when they parsed cleanly with no undefined references and no duplicate declarations. + +After dedup against the held-out benchmark gold and chat formatting: ~3,700 train / ~460 valid pairs. + +## Inference + +### CLI (recommended) + +```bash +pip install stackfix # installs the `taxify` command +taxify "a Customer model with id and email" +``` + +### Python (llama-cpp-python) + +```python +from llama_cpp import Llama +SYS = ("You translate natural-language requirements into idiomatic Taxi schema code. " + "Taxi is the schema language used by Orbital (orbitalhq.com). " + "Return ONLY the Taxi source.") +llm = Llama(model_path="taxi-nl-3b-q4.gguf", n_ctx=4096, verbose=False) +resp = llm.create_chat_completion(messages=[ + {"role": "system", "content": SYS}, + {"role": "user", "content": "Define a Customer model with id and email"}, +]) +print(resp["choices"][0]["message"]["content"]) +``` + +### With existing Taxi context + +```bash +taxify "Add an Order service that fetches orders by CustomerId" --schema customer.taxi +``` + +The CLI applies a post-processor that strips any blocks the model produces whose declared symbol already exists in the in-context schema — the fine-tune occasionally replays context verbatim, and this dedup keeps output clean. + +## Files + +- `taxi-nl-3b-q4.gguf` — Q4_K_M quantization, **1.8 GB**, recommended for CPU. +- `taxi-nl-3b-f16.gguf` — full f16, 6.2 GB, for evaluation parity. + +## Limitations + +- **Out-of-distribution prompts** (e.g., "design a healthcare claims system using OpenBanking conventions") fall back toward the base model's behaviour. Best for "translate this concrete schema description" rather than open-ended design. +- **Reserved word handling** is imperfect — names like `from` as parameter names occasionally trip the parser. +- **TaxiQL projections** (`find { X[] } as { ... }`) work for simple cases; complex projection rewrites are still best-effort. +- The post-process dedup is a heuristic (exact symbol-name match). If you genuinely want the schema repeated, set `--no-dedup` (CLI) or skip the helper. + +## License + +Apache 2.0 (matching the base model's license). + +## Citation + +```bibtex +@misc{taxi-nl-3b, + author = {Cloud-Gym}, + title = {taxi-nl-3b: a small fine-tuned model for NL→Taxi translation}, + year = {2026}, + publisher = {HuggingFace}, +} +``` diff --git a/taxi-nl-3b-f16.gguf b/taxi-nl-3b-f16.gguf new file mode 100644 index 0000000..fd25f15 --- /dev/null +++ b/taxi-nl-3b-f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:374cd8db1b16e67be02bacc9a8bafc0042177bc9dbea458bbeaa881fe1ddf5d0 +size 6178317280 diff --git a/taxi-nl-3b-q4.gguf b/taxi-nl-3b-q4.gguf new file mode 100644 index 0000000..30d7c98 --- /dev/null +++ b/taxi-nl-3b-q4.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c49a3410eed1926b08047faf918808bfe3db9b0bed7a440b7dc31f43d6025e3 +size 1929903072