47 lines
1.5 KiB
Markdown
47 lines
1.5 KiB
Markdown
---
|
||
library_name: transformers
|
||
license: other
|
||
base_model: Qwen/Qwen3-1.7B
|
||
tags:
|
||
- llama-factory
|
||
- full
|
||
- generated_from_trainer
|
||
- routing
|
||
- classifier
|
||
model-index:
|
||
- name: societas-router-lenv3-17b
|
||
results: []
|
||
---
|
||
|
||
# societas-router-lenv3-17b
|
||
|
||
A per-call **LLM routing classifier** — a full-parameter fine-tune of
|
||
[Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B). At each step of an agent's trajectory it
|
||
reads a read-only snapshot of the trajectory-so-far and predicts the capability **tier** the *next*
|
||
model call needs, emitting a single JSON verdict `{"tier": "...", "reason": "..."}` with
|
||
`tier ∈ {Routine, Medium, Advanced}`.
|
||
|
||
## Results (held-out test split, 251 examples)
|
||
|
||
- **Accuracy: 60.16%** (151/251), `parse_fail = 0`, no think-token leakage. (chance = 33%)
|
||
- Per-tier recall: Routine 58.3% · Medium 52.4% · Advanced 69.9%.
|
||
|
||
## Usage note (important)
|
||
|
||
Trained with the Qwen3 chat template at `enable_thinking=False` (the empty `<think></think>` block is
|
||
in the prompt; the JSON verdict is the target). **Serve it the same way** — render prompts with
|
||
`enable_thinking=False`.
|
||
|
||
## Training procedure
|
||
|
||
Full-parameter SFT via LLaMA-Factory, bf16, on 2× A800-80G.
|
||
|
||
- learning_rate: 2e-05, cosine schedule, warmup_ratio 0.03
|
||
- per_device_train_batch_size: 1, gradient_accumulation_steps: 8, total_train_batch_size: 16
|
||
- num_epochs: 3.0, cutoff_len (max context): 16384, packing: off
|
||
- optimizer: AdamW (torch), betas=(0.9,0.999), eps=1e-08
|
||
|
||
### Framework versions
|
||
|
||
- Transformers 4.53.x / Pytorch 2.5.1+cu121 / Datasets / Tokenizers
|