初始化项目,由ModelHub XC社区提供模型
Model: ayh015/myLightningOPD Source: Original Platform
This commit is contained in:
50
slime_plugins/rollout_buffer/README.md
Normal file
50
slime_plugins/rollout_buffer/README.md
Normal file
@@ -0,0 +1,50 @@
|
||||
# Rollout Buffer
|
||||
|
||||
## Overview
|
||||
|
||||
Rollout Buffer is an independent component for asynchronous agent trajectory generation, with the main function of using the LLM OpenAI Server launched by slime training to generate agent trajectories.
|
||||
|
||||
### Workflow
|
||||
|
||||
```
|
||||
slime Training Process ←─── HTTP API ───→ Rollout Buffer
|
||||
↓ ↓
|
||||
LLM Server ←─────── HTTP Requests ─────── Agent Framework
|
||||
↓ ↓
|
||||
Model Response ──────────────────────→ Trajectory Generation
|
||||
```
|
||||
|
||||
For each different Agent task, there should be a corresponding independent Generator class, responsible for generating trajectories for that type of task. Rollout Buffer automatically reads and loads different types of Generators.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Basic Usage Process
|
||||
|
||||
1. **Copy Template**: Copy `base_generator.py` as a template
|
||||
2. **Modify Task Type**: Change `TASK_TYPE` to your task name (cannot duplicate with other Generators)
|
||||
3. **Implement Core Function**: Implement the `run_rollout()` function
|
||||
4. **Optional Customization**: Rewrite five optional functions as needed
|
||||
|
||||
|
||||
Generator files must end with `_generator.py` and be placed in the `generator/` directory:
|
||||
|
||||
```
|
||||
generator/
|
||||
├── base_generator.py # Math task implementation (default template)
|
||||
└── your_task_generator.py # Your custom task
|
||||
```
|
||||
|
||||
Each Generator file must define `TASK_TYPE` and `run_rollout()`.
|
||||
|
||||
In addition, Rollout Buffer also provides some customizable functions to meet special needs of different tasks. If no custom implementation is provided, the system will use default implementations (located in `slime_plugins/rollout_buffer/default_func.py`).
|
||||
|
||||
### Example Script
|
||||
|
||||
First, you need to follow [Example: Qwen3-4B Model](../../docs/en/models/qwen3-4B.md) to configure the environment, download data and convert model checkpoints. And then run the following scripts:
|
||||
```bash
|
||||
cd slime_plugins/rollout_buffer
|
||||
bash rollout_buffer_example.sh
|
||||
|
||||
# In a different terminal
|
||||
python buffer.py
|
||||
```
|
||||
Reference in New Issue
Block a user