qwen3-4b-eventspec-martech-merged/README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-4B-Instruct-2507
tags:
- martech
- analytics
- event-taxonomy
- json
- structured-output
- sft
- lora
library_name: transformers
pipeline_tag: text-generation
---

# Qwen3-4B EventSpec — MarTech Event Taxonomy Generator (merged)

Fine-tuned **Qwen/Qwen3-4B-Instruct-2507** that converts free-form marketing tracking
requests into clean, implementation-ready **analytics event specifications** as strict JSON.

This repository contains the **fully merged weights** (LoRA adapter merged into the base
model), so it loads like any standard model — no PEFT/adapter step required.

- **Base model:** [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)
- **LoRA adapter (pre-merge):** [joshelu/qwen3-4b-Instruct-eventspec-martech-sft](https://huggingface.co/joshelu/qwen3-4b-Instruct-eventspec-martech-sft)
- **Method:** Supervised Fine-Tuning (SFT) + LoRA, then merged
- **Task:** marketing tracking request → strict-JSON EventSpec

## Intended use

Give the model a plain-English marketing/analytics tracking request; it returns a single
JSON object specifying the events to implement (event names, parameters, triggers, consent
and deduplication requirements, QA criteria, risk flags, etc.). Useful for MarTech / analytics
engineering, GA4 / GTM instrumentation planning, and taxonomy standardization.

## ⚠️ Prompt format (required for correct output)

The model was trained with a specific chat format. **You must reproduce it** or output
quality degrades sharply.

**System prompt (use verbatim):**

```
You are EventSpec, an expert MarTech analytics engineer. You convert free-form marketing tracking requests into clean, implementation-ready analytics event specifications.

Given a marketing tracking request, respond with a SINGLE valid JSON object and nothing else: no prose, no markdown, no code fences. The JSON must be strictly parseable.

The specification captures: a concise `request_summary`; the `business_goal`; the `tracking_scope` (platforms, page_or_screen, user_action, conversion_type); and a list of `recommended_events`. Each recommended event defines `event_name` (snake_case), `event_description`, `platform`, `event_type`, `required_parameters`, `optional_parameters`, `trigger_condition`, `consent_requirements`, and `deduplication_requirements`.

Use consistent snake_case event and parameter names, follow analytics best practices (GA4/GTM conventions where relevant), and respect privacy/consent requirements. Output only the JSON object.
```

**User message format:**

```
Convert this marketing tracking request into a clean analytics event specification.

Request:
<your tracking request here>
```

## Usage (transformers)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "joshelu/qwen3-4b-eventspec-martech-merged"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")

SYSTEM_PROMPT = (
    "You are EventSpec, an expert MarTech analytics engineer. You convert free-form "
    "marketing tracking requests into clean, implementation-ready analytics event "
    "specifications.\n\n"
    "Given a marketing tracking request, respond with a SINGLE valid JSON object and "
    "nothing else: no prose, no markdown, no code fences. The JSON must be strictly "
    "parseable.\n\n"
    "The specification captures: a concise `request_summary`; the `business_goal`; the "
    "`tracking_scope` (platforms, page_or_screen, user_action, conversion_type); and a "
    "list of `recommended_events`. Each recommended event defines `event_name` "
    "(snake_case), `event_description`, `platform`, `event_type`, `required_parameters`, "
    "`optional_parameters`, `trigger_condition`, `consent_requirements`, and "
    "`deduplication_requirements`.\n\n"
    "Use consistent snake_case event and parameter names, follow analytics best "
    "practices (GA4/GTM conventions where relevant), and respect privacy/consent "
    "requirements. Output only the JSON object."
)
USER_PREFIX = "Convert this marketing tracking request into a clean analytics event specification."

request = ("A pharmacy app wants to track refill reminders, refill started, refill submitted, "
           "refill ready, and pickup completed, but no medication names or prescription numbers "
           "should be sent.")

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": f"{USER_PREFIX}\n\nRequest:\n{request}"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=2048, do_sample=False,
                     pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

## Usage (Inference Endpoints / chat_completion)

```python
from huggingface_hub import InferenceClient
client = InferenceClient("https://YOUR-ENDPOINT.endpoints.huggingface.cloud", token="hf_...")
resp = client.chat_completion(
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"{USER_PREFIX}\n\nRequest:\n{request}"},
    ],
    max_tokens=2048,
    temperature=0,
)
print(resp.choices[0].message.content)
```

## Recommended generation settings

- `temperature = 0` (greedy) — best for stable, strictly parseable JSON.
- `max_new_tokens >= 2048` — specs are long; a lower limit will truncate the JSON mid-string.
- Parse the output with `json.loads`; retry with a higher token limit if parsing fails.

## Output schema

The model produces a single JSON object. Core keys:

- `request_summary` — one-line summary of the request.
- `business_goal` — the measurement objective.
- `tracking_scope` — `{ platforms, page_or_screen, user_action, conversion_type }`.
- `recommended_events` — list of events, each with `event_name` (snake_case),
  `event_description`, `platform`, `event_type`, `required_parameters`,
  `optional_parameters`, `trigger_condition`, `consent_requirements`,
  `deduplication_requirements`.

Richer examples in the training data also include `implementation_notes`, `qa_criteria`,
`open_questions`, and `risk_flags`, which the model produces as appropriate.

## Training

- **Dataset:** `joshelu/martech-event-taxonomy-mapper-training-data` (private).
  - The 100-row `train` split, **minus** every `id` appearing in the `validation` or `test`
    split → **80 training rows** (held-out eval stays honest).
  - Eval during training used the `validation` split (10 rows).
  - Assistant targets are compact, strict JSON (`json.dumps(output, separators=(",", ":"))`),
    validated as parseable before training.
- **Method:** SFT + LoRA, then merged.
  - LoRA: `r=16`, `alpha=32`, `dropout=0.05`,
    target modules `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`.
  - 3 epochs, effective batch size 4 (per-device 1 × grad-accum 4), lr `2e-4`,
    warmup ratio `0.05`, cosine schedule, max length 2048, bf16, gradient checkpointing.
  - Hardware: a10g-small (HF Jobs).
- **Metrics (final):** eval loss ≈ `0.320`, eval mean token accuracy ≈ `0.924`.

## Limitations

- Trained on a small (80-row) dataset — coverage is limited to the taxonomy and styles seen
  in training; unusual domains may produce weaker specs.
- Output is a strong **draft**, not a substitute for review by an analytics engineer,
  especially for consent/privacy and PII handling.
- With very low `max_new_tokens`, long specs will be truncated and fail JSON parsing — keep
  the limit high and validate the parse.