54 lines
2.6 KiB
Markdown
54 lines
2.6 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model: Qwen/Qwen3-8B
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- agent
|
||
|
|
- multi-turn
|
||
|
|
- reinforcement-learning
|
||
|
|
---
|
||
|
|
|
||
|
|
# Qwen3-8B-Base-Agent
|
||
|
|
|
||
|
|
The Base agent from MIMESIS: Qwen3-8B after a short supervised warm start, before multi-turn reinforcement learning. The RL agents in the collection all start from this checkpoint.
|
||
|
|
|
||
|
|
[Paper](https://arxiv.org/abs/2610.09484) · [Code](https://github.com/VietHoang1512/mimesis) · [Project page](https://viethoang1512.github.io/mimesis/) · [Collection](https://huggingface.co/collections/phanviethoang1512/mimesis)
|
||
|
|
|
||
|
|
## Results
|
||
|
|
|
||
|
|
Mean task score of the five Qwen3-8B agents from the paper across eight UserRL environments, three of them held out from training (IntentionGym, TelepathyGym, SearchGym), under nine evaluation user models that were not used in training.
|
||
|
|
|
||
|
|
| Agent | Training user | Objective | Mean score |
|
||
|
|
|---|---|---|---:|
|
||
|
|
| **Base (this model)** | none | SFT | 18.49 |
|
||
|
|
| [UserRL](https://huggingface.co/phanviethoang1512/Qwen3-8B-UserRL-Agent) | GPT-5.5 | GRPO | 26.10 |
|
||
|
|
| [InfoPO](https://huggingface.co/phanviethoang1512/Qwen3-8B-InfoPO-Agent) | MIMESIS-9B | InfoPO | 26.65 |
|
||
|
|
| [UserRL+](https://huggingface.co/phanviethoang1512/Qwen3-8B-UserRL-Plus-Agent) | MIMESIS-9B | GRPO | 29.54 |
|
||
|
|
| [CSD](https://huggingface.co/phanviethoang1512/Qwen3-8B-CSD-Agent) | MIMESIS-9B | GRPO + CSD | **31.09** |
|
||
|
|
|
||
|
|
Each score is the mean over the nine evaluation users; per-user and per-environment scores are in Appendix D of the paper.
|
||
|
|
|
||
|
|
## Training
|
||
|
|
|
||
|
|
Full fine-tuning of Qwen3-8B on about 1,000 multi-turn trajectories from the five training environments (TravelGym, TurtleGym, FunctionGym, TauGym and PersuadeGym). The trajectories were generated with GPT-4o playing both the agent and the user, keeping the top-ranked ones in each environment, and each agent turn is reasoning in `<think>` tags followed by one tool call.
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
The model loads like Qwen3-8B. To evaluate it in the eight environments, install the [agent code](https://github.com/VietHoang1512/mimesis/tree/main/agent), serve a user model as described there, and run from the `agent` directory:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
bash scripts/run_eval.sh phanviethoang1512/Qwen3-8B-Base-Agent base
|
||
|
|
```
|
||
|
|
|
||
|
|
## Citation
|
||
|
|
|
||
|
|
```bibtex
|
||
|
|
@article{phan2026mimesis,
|
||
|
|
title = {{MIMESIS}: Learning User Simulators as Training Environments for Interactive Agents},
|
||
|
|
author = {Phan, Hoang and Huynh, Dat and Zhmoginov, Andrey and Zeng, Qi and Mu, Wancen and Cao, Yue and Bi, Shengjie and He, Yun and Oh, Changdae and Lei, Deren},
|
||
|
|
journal = {arXiv preprint arXiv:2610.09484},
|
||
|
|
year = {2026}
|
||
|
|
}
|
||
|
|
```
|