69 lines
2.8 KiB
Markdown
69 lines
2.8 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model: Qwen/Qwen3-4B-Base
|
|||
|
|
tags:
|
|||
|
|
- event-merge
|
|||
|
|
- crimeradar
|
|||
|
|
- sft
|
|||
|
|
- reasoning
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# crimeradar-event-merge-qwen3-4b
|
|||
|
|
|
|||
|
|
A **Qwen3-4B-Base** student, full fine-tuned (no LoRA) to do **event_merge** for
|
|||
|
|
CrimeRadar: given a set of candidate events (ASR-derived dispatch/incident reports with
|
|||
|
|
id, summary and segment timelines), decide which ones refer to the **same real-world
|
|||
|
|
incident** and emit the merged grouping. Distilled from a dual-teacher trajectory set
|
|||
|
|
(GLM-5.2 + minimax) on the private dataset `NewsBreak/crimeradar-event-merge` (flat
|
|||
|
|
`train` split, both teachers). 4B counterpart of the qwen3-8b reasoning model.
|
|||
|
|
|
|||
|
|
## Output format
|
|||
|
|
|
|||
|
|
The model is a reasoning model: it produces a `<think>...</think>` block, then a JSON
|
|||
|
|
answer with `events_csv` (and `groups`). The chat template places the opening `<think>`
|
|||
|
|
in the prompt, so generation begins inside the reasoning block and the model emits the
|
|||
|
|
closing `</think>` followed by the answer.
|
|||
|
|
|
|||
|
|
## Eval (60-item consensus benchmark, vs 3-judge consensus gold)
|
|||
|
|
|
|||
|
|
Pairwise P/R/F1 + ARI, sampling temp 0.8 / top_p 0.95 / top_k 40 / presence_penalty 0.5.
|
|||
|
|
|
|||
|
|
| model | F1 | P | R |
|
|||
|
|
|---|---|---|---|
|
|||
|
|
| **this model (8-seed mean)** | **0.768 ± 0.042** | 0.752 | 0.790 |
|
|||
|
|
| teacher gpt-5-chat | 0.765 | 0.672 | 0.886 |
|
|||
|
|
| prior Qwen3-8B v2 SFT | 0.630 | 0.793 | 0.523 |
|
|||
|
|
| prior Qwen3-4B v1 SFT | 0.571 | — | — |
|
|||
|
|
|
|||
|
|
A 4B student at teacher level on F1, ≫ the prior best 8B SFT (0.630). The larger
|
|||
|
|
qwen3-8b sibling reaches F1 0.824. Best single seed here: P 0.812 / R 0.886 / F1 0.848.
|
|||
|
|
|
|||
|
|
## Recommended inference (vLLM)
|
|||
|
|
|
|||
|
|
Reliability-first sampling: `temperature=0.8, top_p=0.95, top_k=40, presence_penalty=0.5,
|
|||
|
|
max_tokens=32768`, stop on `<|im_end|>`. (These are baked into `generation_config.json`;
|
|||
|
|
`presence_penalty` is the one thing to pass at request time.)
|
|||
|
|
|
|||
|
|
A rare tail of the largest inputs (n_events≈11–15, with shared ASR segments / ambiguous
|
|||
|
|
boundaries) can run the `<think>` block to the token cap without closing it (~1% of
|
|||
|
|
generations). To bound it, serve with vLLM and a per-request **thinking-token budget** —
|
|||
|
|
the qwen3 reasoning parser already knows `<think>`/`</think>`:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
vllm serve <this-repo> --reasoning-parser qwen3
|
|||
|
|
# per request: SamplingParams(thinking_token_budget=16384) # offline
|
|||
|
|
# extra_body={"thinking_token_budget": 16384} # OpenAI API
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
This clips only the runaway loops and leaves normal reasoning untouched; F1 is unchanged.
|
|||
|
|
|
|||
|
|
## Training
|
|||
|
|
|
|||
|
|
Full fine-tune, ZeRO-2, 8×GPU. 3 epochs (639 steps), lr 6e-5 cosine, warmup 0.05,
|
|||
|
|
effective batch 64 (per-device 1 × grad-accum 8 × 8 gpu), max_len 65536, bf16/tf32,
|
|||
|
|
flash-attention-2, gradient checkpointing, liger kernel. Final train loss ≈ 0.067.
|
|||
|
|
Trained on large-scene-enriched data (≥16-event inputs ~32% of training rows).
|