53 lines
2.1 KiB
Markdown
53 lines
2.1 KiB
Markdown
---
|
||
library_name: transformers
|
||
tags:
|
||
- reinforcement-learning
|
||
- retrieval
|
||
- search-agent
|
||
- s3
|
||
- on-policy
|
||
- research
|
||
model-index:
|
||
- name: s3-8-3-3-20steps
|
||
results: []
|
||
---
|
||
|
||
# s3-8-3-3-20steps
|
||
|
||
**s3** is a reinforcement-learning–trained search agent that learns to plan retrieval and answer questions efficiently.
|
||
This release provides weights **for research replication** only. For usage, training, and evaluation **follow our GitHub repo** (we intentionally do not include inference snippets here).
|
||
|
||
- 📄 Reference: *“s3: You Don’t Need That Much Data to Train a Search Agent via RL”* (EMNLP 2025 Main).
|
||
- 🧑💻 GitHub: https://github.com/pat-jj/s3
|
||
|
||
## What is in this repo?
|
||
|
||
A Hugging Face model folder with tokenizer files and sharded `*.safetensors` checkpoints exported from our VERL training runs (the “actor” policy).
|
||
File layout mirrors the training outputs (e.g., `config.json`, `tokenizer.json`, and `model-00001-of-00004.safetensors`, etc.).
|
||
|
||
## Important notes
|
||
|
||
- **We highly recommend training the model yourself via the GitHub repo.** In our experience, **testing/inference time can be much much heavier than training time**.
|
||
- **Do not** treat these weights as a drop-in general QA system; they are intended for the s3 pipelines described in the paper and codebase.
|
||
- **To run anything**, please **follow the GitHub instructions** end-to-end (env setup, datasets, evaluation scripts, and RL configs).
|
||
|
||
|
||
|
||
## Intended use & limitations
|
||
|
||
Research replication, ablations, and educational study of on-policy RL for retrieval-augmented search agents.
|
||
Commercial or safety-critical use is **not** advised without extensive review and additional safeguards.
|
||
|
||
## Citation
|
||
```bibtex
|
||
@inproceedings{jiang2025s3,
|
||
title = {s3: You Don't Need That Much Data to Train a Search Agent via RL},
|
||
author = {Jiang, Pengcheng and Xu, Xueqiang and Lin, Jiacheng and Xiao, Jinfeng and Wang, Zifeng and Sun, Jimeng and Han, Jiawei},
|
||
year = {2025},
|
||
booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
|
||
}
|
||
```
|
||
|
||
|
||
*Last updated:* 2025-09-29
|