104 lines
4.3 KiB
Markdown
104 lines
4.3 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model:
|
|
- Qwen/Qwen2.5-3B-Instruct
|
|
language:
|
|
- en
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- reinforcement-learning
|
|
- search-agent
|
|
- rag
|
|
- memory
|
|
- grpo
|
|
---
|
|
|
|
# MemSearcher-3B
|
|
|
|
<p align="center">
|
|
📄 <a href="https://arxiv.org/abs/2511.02805">Paper</a> |
|
|
💻 <a href="https://github.com/icip-cas/MemSearcher">Code</a>
|
|
</p>
|
|
|
|
## Model Summary
|
|
|
|
**MemSearcher-3B** is a search agent built on [`Qwen/Qwen2.5-3B-Instruct`]( https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) and trained with
|
|
**multi-context GRPO**, the end-to-end reinforcement-learning method introduced in
|
|
[MemSearcher](https://arxiv.org/abs/2511.02805).
|
|
|
|
Unlike ReAct-style agents that concatenate the *entire* interaction history into the context — producing long,
|
|
noisy inputs and linearly growing compute — MemSearcher maintains a **compact, iteratively-updated memory**.
|
|
At every turn the backbone LLM sees only two short inputs, the **question** and the current **memory**, then it
|
|
reasons, decides whether to search, and rewrites the memory to keep only question-relevant information. This keeps
|
|
the context length **nearly constant across turns** without sacrificing accuracy.
|
|
|
|
## Evaluation
|
|
|
|
Exact Match (EM) on seven single- and multi-hop QA benchmarks (Wikipedia-18 corpus):
|
|
|
|
| Model | NQ | TriviaQA | PopQA | HotpotQA | 2Wiki | MuSiQue | Bamboogle | Avg. |
|
|
|---|---|---|---|---|---|---|---|---|
|
|
| **MemSearcher 3B** | 47.0 | 63.8 | 47.9 | 43.9 | 43.5 | 17.9 | 42.4 | **43.8** |
|
|
| MemSearcher 7B | 52.7 | 68.1 | 47.8 | 50.8 | 48.6 | 25.8 | 48.8 | 48.9 |
|
|
| MemSearcher 14B | 53.7 | 71.1 | 48.8 | 51.8 | 51.5 | 27.2 | 57.6 | 51.7 |
|
|
|
|
This card is for the **3B** model (highlighted row).
|
|
|
|
## Key Features
|
|
|
|
- 🧠 **Compact memory** — keeps only question-relevant information instead of the full history.
|
|
- 📏 **Near-constant context** — token count stays stable across multi-turn interactions, cutting compute/memory.
|
|
- 🎯 **Multi-context GRPO** — propagates trajectory-level advantages to every turn for true end-to-end RL.
|
|
- 🔍 **Strong QA performance** — outperforms strong history-concatenation (ReAct-style) baselines across 7 benchmarks.
|
|
|
|
## How It Works
|
|
|
|
At turn *i* the context is just `(question, memory_{i-1})`. The model:
|
|
1. **Reasons** inside `<think>...</think>`;
|
|
2. **Searches** when it lacks knowledge (a `wikipedia_search` tool call), or answers in `\boxed{}` when the memory suffices;
|
|
3. **Updates the memory** from the retrieved results, retaining only what helps answer the question.
|
|
|
|
Training uses **multi-context GRPO**: it samples a group of trajectories, then propagates each trajectory's
|
|
advantage to all of its per-turn contexts, enabling stable end-to-end optimization of reasoning, search, and
|
|
memory management.
|
|
|
|
## Quickstart
|
|
|
|
MemSearcher is an **agent** that needs a Wikipedia retriever and the multi-turn orchestration loop. The simplest
|
|
path is the released code:
|
|
|
|
```bash
|
|
git clone https://github.com/icip-cas/MemSearcher && cd MemSearcher
|
|
# 1) start the wiki-18 retriever (see README "Retriever Serving")
|
|
# 2) serve this model with SGLang
|
|
MODEL_PATH=yuanqianhao/MemSearcher-3B TP=2 PORT=80 bash launch_server.sh
|
|
# 3) run the agentic evaluation
|
|
GENERATOR_MODEL=yuanqianhao/MemSearcher-3B DATA_DIR=./data SAVE_DIR=./eval \
|
|
SGL_REMOTE_URL=http://127.0.0.1:80 RETRIEVER_URL=http://127.0.0.1:8000/search bash eval.sh
|
|
```
|
|
|
|
The model weights load with standard `transformers` (`AutoModelForCausalLM.from_pretrained("yuanqianhao/MemSearcher-3B")`),
|
|
but reproducing the reported numbers requires the retriever + agent loop above.
|
|
|
|
## Training Data
|
|
|
|
Trained on the **NQ + HotpotQA** training split — the same data as
|
|
[Search-R1](https://github.com/PeterGriffinJin/Search-R1) — re-wrapped into MemSearcher's re_call format.
|
|
See the code repository for the exact preparation script.
|
|
|
|
## Citation
|
|
|
|
```bibtex
|
|
@article{yuan2025memsearcher,
|
|
title={MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning},
|
|
author={Yuan, Qianhao and Lou, Jie and Li, Zichao and Chen, Jiawei and Lu, Yaojie and Lin, Hongyu and Sun, Le and Zhang, Debing and Han, Xianpei},
|
|
journal={arXiv preprint arXiv:2511.02805},
|
|
year={2025}
|
|
}
|
|
```
|
|
|
|
## License
|
|
|
|
Released under Apache-2.0, following the base model [`Qwen/Qwen2.5-3B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct).
|