---
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model: Qwen/Qwen3-4B-Base
base_model_relation: finetune
tags:
- math
- recursive-reasoning
- supervised-fine-tuning
- qwen3
datasets:
- openai/gsm8k
---

# Qwen3-4B Recursive SFT V3.2

This is the **original-tag recursive V3.2 balanced SFT checkpoint**. It is not
V3.1, the JSON variant, the flattened baseline, or an RL-finetuned model.
The weights and tokenizer are unchanged from the final saved checkpoint named
`qwen3-4b-recursive-sft-v3.2-balanced`.

The model was trained from Qwen3-4B-Base with additional recursive protocol
tokens. It was not initialized from the already SFT-trained V3.1 model.
It is a research math-solving model, not a general-purpose chat assistant.

## Download

No ZIP extraction or login is required:

```python
from huggingface_hub import snapshot_download

snapshot_download(
    "tawer12/qwen3-4b-recursive-sft-v3.2",
    local_dir="qwen3-4b-recursive-sft-v3.2",
)
```

The two safetensors shards, tokenizer files, and configuration files must be
kept together. `release_manifest.json` lists file sizes and SHA256 checksums.
The download is approximately 8.06 GB. There are no optimizer states, pickled
training arguments, intermediate checkpoints, training examples, or evaluation
rollouts in this repository.

## Training

V3.2 downsamples only direct-solve examples from the expanded V3.1 call-level
training corpus, retaining the decomposition and aggregation examples unchanged.

| Setting | Value |
| --- | --- |
| Decomposition examples | 4,707 |
| Direct-solve examples | 4,707: 530 root, 4,177 child |
| Aggregation examples | 4,712 |
| Total component-level examples | 14,126 |
| Unique source problems represented | 5,241 |
| Configured epochs | 6 |
| Optimizer updates | 660 |
| Global batch | 128 component calls, not 128 source problems |
| GPUs / per-device batch / accumulation | 8 / 1 / 16 |
| Maximum sequence length | 2,048 tokens |
| Learning rate / weight decay | 1e-5 / 0.01 |
| Warmup / scheduler | 10% / cosine |
| Precision / seed | BF16 / 1 |
| Objective | Output-token-only causal language modeling |

The six epochs approximately match the previous four-epoch V3.1 run's total
example exposure; this is not an exactly token- or update-matched experiment.
The corpus contains independent component calls, not necessarily complete trees
after downsampling. No syntax repair was applied: 387 targets retain known
defects under the newer strict `Final answer:` requirement. The training summary
is included separately from the raw training state.

## Recursive Inference

Inputs are **raw recursive prompt strings**, not Qwen chat-template conversations.
Use `prompts/recursive_sft_prefix_prompt.txt`. Preserve protocol special tokens
when decoding (`skip_special_tokens=False`). A single generation returns one
action, not necessarily the entire solution:

- `SOLVE_DIRECTLY`: solve the supplied problem.
- `DECOMPOSE`: propose at least two child paths for the engine to execute.
- `AGGREGATE`: combine child answers when the engine requests aggregation.

The bundled `recursive_engine.py` provides a standalone, unconstrained baseline
engine. This example executes a complete tree:

```bash
pip install 'transformers==4.51.1' torch accelerate huggingface_hub
```

```python
import sys
from pathlib import Path
from huggingface_hub import snapshot_download

snapshot = Path(snapshot_download("tawer12/qwen3-4b-recursive-sft-v3.2"))
sys.path.insert(0, str(snapshot))
from recursive_engine import RecursiveEngine, load_model_and_tokenizer

model, tokenizer = load_model_and_tokenizer(str(snapshot), dtype="bf16", device_map="auto")
engine = RecursiveEngine(
    model=model,
    tokenizer=tokenizer,
    prefix_template=(snapshot / "prompts/recursive_sft_prefix_prompt.txt").read_text(),
    max_new_tokens=2048,
    temperature=0.0,
    top_p=1.0,
    max_depth=3,
    max_calls=10,
)
result = engine.rollout("What is the smallest multiple of 5 greater than -32?")
print(result.full_trace)
print(result.final_answer, result.parsed_ok, result.parse_errors)
```

For this bundled legacy engine, `max_calls=10` is a **whole-tree model-call limit**,
including the root and aggregations, not ten children per node. The newer
constrained GRPO/evaluation implementation in the
[math-recursive repository](https://github.com/0315jaewon/math-recursive)
has separate per-decomposition and whole-tree limits. Do not conflate them.
The example helper enables the generation cache even though the saved training
configuration has `use_cache=false`.

## Limitations And Attribution

Format validity and answer correctness are distinct. A malformed downstream call
can invalidate a tree, and well-formed child answers can be mathematically wrong.
Decomposition paths may be dependent or underspecified. Aggregation receives
child answer summaries rather than their full reasoning. The standalone engine
executes children sequentially; critical-path token counts are not measured
parallel wall-clock speedups. Grammar constraints are external decoding support,
not an intrinsic guarantee supplied by this checkpoint.

The historical helper's answer extraction is not an authoritative benchmark
grader; use explicit root-answer grading and the standard evaluation protocol
for comparisons. No new benchmark accuracy claims are made in this release.

This is a modified, fine-tuned version of the Qwen team's Apache-2.0
Qwen3-4B-Base model. The upstream license is included in `LICENSE`. GSM8K supplies
the curated training questions. The model card and publishing recipe were added
for this V3.2 release on 2026-09-30.