56 lines
1.6 KiB
Markdown
56 lines
1.6 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen2.5-7B
|
||
tags:
|
||
- retool
|
||
- sft
|
||
- tool-use
|
||
- slime
|
||
- megatron
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# qwen2.5-7b-base-retool-slime-sft-v2
|
||
|
||
ReTool SFT cold-start of **Qwen/Qwen2.5-7B** (base, *not* Instruct), trained with
|
||
[slime](https://github.com/THUDM/slime) on a Megatron backend. This is the tool-format
|
||
SFT stage that teaches the base model the ReTool interleaved code/tool-call format before
|
||
RL.
|
||
|
||
## Training configuration
|
||
|
||
| Setting | Value |
|
||
|---|---|
|
||
| Base model | `Qwen/Qwen2.5-7B` (base) |
|
||
| Dataset | [ReTool-SFT](https://huggingface.co/datasets/JoeYing/ReTool-SFT) (`messages` format) |
|
||
| Loss | `sft_loss`, per-token, Qwen loss mask |
|
||
| **Global batch size** | **32** |
|
||
| **Epochs** | **6** (~371 optimizer steps over the ~2k-row corpus, ~62 steps/epoch) |
|
||
| Optimizer | Adam (β1=0.9, β2=0.95), weight decay 0.01 |
|
||
| LR schedule | 1e-5 → 1e-6, cosine, 10% warmup |
|
||
| Precision | bf16, grads all-reduced in fp32 |
|
||
| Parallelism | TP=4, PP=1, CP=1, sequence-parallel |
|
||
| Hardware | 8× A100-40GB |
|
||
| Final train loss | ~0.02 |
|
||
|
||
The checkpoint corresponds to iteration **371** (end of epoch 6).
|
||
|
||
## Usage
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
"BillyWang1/qwen2.5-7b-base-retool-slime-sft-v2",
|
||
torch_dtype="bfloat16",
|
||
device_map="auto",
|
||
)
|
||
tok = AutoTokenizer.from_pretrained("BillyWang1/qwen2.5-7b-base-retool-slime-sft-v2")
|
||
```
|
||
|
||
## Notes
|
||
|
||
This is an intermediate SFT cold-start checkpoint intended as the starting point for
|
||
subsequent RL (GRPO / GFlow-RL) in the ReTool pipeline.
|