101 lines
3.4 KiB
Markdown
101 lines
3.4 KiB
Markdown
---
|
||
license: apache-2.0
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
language:
|
||
- en
|
||
- zh
|
||
base_model:
|
||
- Qwen/Qwen3-4B-Thinking-2507
|
||
tags:
|
||
- agent
|
||
- deep-research
|
||
- react
|
||
- search
|
||
- sft
|
||
- cold-start
|
||
- thinking
|
||
- LiteResearcher
|
||
datasets:
|
||
- simplex-ai-inc/LiteResearcher-Data
|
||
---
|
||
|
||
# LiteResearcher-4B-SFT
|
||
|
||
This is the **SFT cold-start checkpoint** for [LiteResearcher](https://github.com/simplex-ai-inc/LiteResearcher) —
|
||
a scalable agentic RL training framework for deep-research agents.
|
||
|
||
It is the initial policy used to launch the two-stage curriculum RL training that
|
||
produces the final [`simplex-ai-inc/LiteResearcher-4B`](https://huggingface.co/simplex-ai-inc/LiteResearcher-4B) model.
|
||
|
||
> If you are looking for the **final RL** model, please use
|
||
> [`simplex-ai-inc/LiteResearcher-4B`](https://huggingface.co/simplex-ai-inc/LiteResearcher-4B).
|
||
> If you want to **reproduce the RL training** from scratch, this is the checkpoint you need.
|
||
|
||
## Model details
|
||
|
||
- **Base model**: [`Qwen/Qwen3-4B-Thinking-2507`](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)
|
||
- **Architecture**: `Qwen3ForCausalLM` (36 layers, hidden 2560, 32 heads, GQA 8 KV heads)
|
||
- **Max position embeddings**: 262,144 (RoPE θ = 5,000,000)
|
||
- **Precision**: `bfloat16`
|
||
- **Total params**: ~4B
|
||
- **Training framework**: [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory)
|
||
|
||
## Training recipe
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Stage | SFT (cold-start before RL) |
|
||
| Base model | `Qwen/Qwen3-4B-Thinking-2507` |
|
||
| Dataset | [`simplex-ai-inc/LiteResearcher-Data`](https://huggingface.co/datasets/simplex-ai-inc/LiteResearcher-Data) (~68.2k SFT trajectories) |
|
||
| Max sequence length | 64K (`cutoff_len=65536`) |
|
||
| Global batch size | 128 (per-device bs 2 × grad-accum 8 × 8 GPUs) |
|
||
| Epochs | 1 |
|
||
| Optimizer steps | 533 |
|
||
| Learning rate | 2.0e-5, cosine, 10% warmup |
|
||
| Final train loss | ≈ 0.447 (starting loss ≈ 1.19) |
|
||
|
||
The SFT trajectories teach the model the ReAct `think` → `search` → `visit` → `answer`
|
||
loop and the strict `<answer>...</answer>` output contract used by the RL environment.
|
||
Because the base is the **Thinking-2507** variant, the model preserves long
|
||
chain-of-thought behavior inside `<think>...</think>` blocks, which is what the
|
||
downstream RL curriculum builds on.
|
||
|
||
## How to use
|
||
|
||
### As the initial policy for RL (recommended use)
|
||
|
||
```bash
|
||
# In the LiteResearcher training scripts (Training/ folder of the repo)
|
||
export MODEL_PATH=$(hf download simplex-ai-inc/LiteResearcher-4B-SFT \
|
||
--local-dir ./literesearcher_sft)
|
||
```
|
||
|
||
Then follow the Stage-1 / Stage-2 RL instructions in the
|
||
[LiteResearcher repository](https://github.com/simplex-ai-inc/LiteResearcher).
|
||
|
||
### Stand-alone inference
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
import torch
|
||
|
||
model_id = "simplex-ai-inc/LiteResearcher-4B-SFT"
|
||
tok = AutoTokenizer.from_pretrained(model_id)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_id, torch_dtype=torch.bfloat16, device_map="auto"
|
||
)
|
||
```
|
||
|
||
The model expects the same ReAct system prompt and tool schema used by
|
||
LiteResearcher (see `Inference/` in the repo).
|
||
|
||
## Citation
|
||
|
||
If you use this checkpoint in academic work, please cite the LiteResearcher
|
||
project — see the [GitHub README](https://github.com/simplex-ai-inc/LiteResearcher) for the BibTeX entry.
|
||
|
||
## License
|
||
|
||
Apache-2.0, inheriting from the [Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507) base model.
|