102 lines
5.8 KiB
Markdown
102 lines
5.8 KiB
Markdown
---
|
||
base_model:
|
||
- Qwen/Qwen3-8B
|
||
datasets:
|
||
- open-thoughts/OpenThoughts-Agent-SFT-ColdStartForRL-10K
|
||
library_name: transformers
|
||
license: apache-2.0
|
||
model-index:
|
||
- name: OpenThinkerAgent-8B-ColdStartSFTForRL
|
||
results: []
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- agents
|
||
- terminal
|
||
- code
|
||
- software-engineering
|
||
- sft
|
||
- cold-start
|
||
---
|
||
<p align="center">
|
||
<img src="https://huggingface.co/datasets/open-thoughts/OpenThoughts1-Agent-SFT/resolve/main/ota-logo.png" width="50%">
|
||
</p>
|
||
|
||
<p align="center">
|
||
<a href="https://www.openthoughts.ai/blog/agent" style="margin-right: 24px;">Project</a> |
|
||
<a href="https://github.com/open-thoughts/OpenThoughts-Agent" style="margin-right: 24px; margin-left: 24px;">Code</a> |
|
||
<a href="https://huggingface.co/collections/open-thoughts/openthinker-agent" style="margin-left: 24px;">Collection</a>
|
||
</p>
|
||
|
||
|
||
# OpenThinkerAgent-8B-ColdStartSFTForRL
|
||
|
||
**OpenThoughts-Agent** is an open-source effort to curate the best datasets for training agents. Our release includes [datasets](https://huggingface.co/collections/open-thoughts/openthinker-agent), [models](https://huggingface.co/collections/open-thoughts/openthinker-agent) and our [research codebase](https://github.com/open-thoughts/OpenThoughts-Agent).
|
||
|
||
[OpenThinkerAgent-8B-ColdStartSFTForRL](https://huggingface.co/open-thoughts/OpenThinkerAgent-8B-ColdStartSFTForRL) is the **cold-start, pre-RL base** of the OpenThoughts-Agent 8B SFT→RL recipe. It is post-trained from [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) with full-parameter SFT on the cold-start [OpenThoughts-Agent-SFT-ColdStartForRL-10K](https://huggingface.co/datasets/open-thoughts/OpenThoughts-Agent-SFT-ColdStartForRL-10K) dataset. Its purpose is to give the model the agentic interaction format and tool-use behaviour needed to make subsequent reinforcement learning stable; it is then RL-trained to produce [OpenThinkerAgent-8B-RL](https://huggingface.co/open-thoughts/OpenThinkerAgent-8B-RL).
|
||
|
||
> **Architecture note.** Although the upstream artifact carries a `GLM-4.7` label (which refers to the *teacher* that generated the SFT trajectories, not the student), this model is a **Qwen3-8B**. Its `config.json` reports `model_type: qwen3`, `architectures: ["Qwen3ForCausalLM"]`, 36 layers, hidden size 4096, 32 attention heads / 8 KV heads, and a 40,960-token context — i.e. standard Qwen3-8B.
|
||
|
||
- **Homepage:** https://www.openthoughts.ai/blog/agent
|
||
- **Repository:** https://github.com/open-thoughts/OpenThoughts-Agent
|
||
|
||
# Model details
|
||
|
||
- **Base model:** [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
|
||
- **Architecture:** Qwen3 (`Qwen3ForCausalLM`), 36 layers, hidden size 4096, 32 attention heads, 8 KV heads, RoPE θ = 1e6
|
||
- **Context length:** 40,960 tokens (max position embeddings)
|
||
- **Vocabulary:** 151,936 tokens
|
||
- **Precision:** bf16
|
||
- **Role in pipeline:** cold-start SFT checkpoint (pre-RL base)
|
||
|
||
# Position in the SFT → RL recipe
|
||
|
||
1. [OpenThoughts-Agent-SFT-ColdStartForRL-10K](https://huggingface.co/datasets/open-thoughts/OpenThoughts-Agent-SFT-ColdStartForRL-10K) — cold-start SFT trajectories.
|
||
2. **[OpenThinkerAgent-8B-ColdStartSFTForRL](https://huggingface.co/open-thoughts/OpenThinkerAgent-8B-ColdStartSFTForRL)** — this model (Qwen3-8B after cold-start SFT, the pre-RL base).
|
||
3. [OpenThoughts-Agent-RL-5K](https://huggingface.co/datasets/open-thoughts/OpenThoughts-Agent-RL-5K) — on-policy RL tasks.
|
||
4. [OpenThinkerAgent-8B-RL](https://huggingface.co/open-thoughts/OpenThinkerAgent-8B-RL) — the final RL'd checkpoint (step 45).
|
||
|
||
# Training data
|
||
|
||
Trained on [OpenThoughts-Agent-SFT-ColdStartForRL-10K](https://huggingface.co/datasets/open-thoughts/OpenThoughts-Agent-SFT-ColdStartForRL-10K) (**9,437** (task, trajectory) pairs): SWE-Smith sandboxed coding tasks with tests, solved by a teacher model in the **terminus-2** harness inside Daytona sandboxes, oracle-verified (120s verifier timeout).
|
||
|
||
# Training procedure
|
||
|
||
Full-parameter SFT (LLaMA-Factory). Hyperparameters as recorded by the trainer:
|
||
|
||
- learning_rate: 4e-05
|
||
- lr_scheduler_type: cosine, warmup_ratio 0.1
|
||
- train_batch_size: 1 per device × 8 devices × gradient_accumulation_steps 2 → total_train_batch_size 16
|
||
- optimizer: AdamW (fused), betas (0.9, 0.98), eps 1e-08
|
||
- num_epochs: 7
|
||
- seed: 42
|
||
- precision: bf16
|
||
- final train loss: ≈ 0.303 (4,130 global steps)
|
||
|
||
### Framework versions
|
||
- Transformers 4.57.6
|
||
- PyTorch 2.9.0+cu128
|
||
- Datasets 4.4.1
|
||
- Tokenizers 0.22.2
|
||
|
||
# Intended uses & limitations
|
||
|
||
This checkpoint is intended as the **starting point for agentic RL**, not as a final deployable agent. It has learned the agentic format and tool-use conventions of the terminus-2 harness from a relatively small cold-start set; its standalone agentic performance is expected to be below the RL-trained successor [OpenThinkerAgent-8B-RL](https://huggingface.co/open-thoughts/OpenThinkerAgent-8B-RL). As with the base Qwen3-8B, outputs may be incorrect or unsafe and should not be executed without review. No standalone agentic-benchmark numbers are published for this cold-start checkpoint.
|
||
|
||
# Links
|
||
- 🌐 [OpenThoughts-Agent project page](https://www.openthoughts.ai/blog/agent)
|
||
- 💻 [OpenThoughts-Agent GitHub repository](https://github.com/open-thoughts/OpenThoughts-Agent)
|
||
- 📚 [OpenThinker-Agent collection](https://huggingface.co/collections/open-thoughts/openthinker-agent)
|
||
- 🧠 [Training dataset: OpenThoughts-Agent-SFT-ColdStartForRL-10K](https://huggingface.co/datasets/open-thoughts/OpenThoughts-Agent-SFT-ColdStartForRL-10K)
|
||
- 🧠 [RL tasks: OpenThoughts-Agent-RL-5K](https://huggingface.co/datasets/open-thoughts/OpenThoughts-Agent-RL-5K)
|
||
- 🤖 [Final RL model: OpenThinkerAgent-8B-RL](https://huggingface.co/open-thoughts/OpenThinkerAgent-8B-RL)
|
||
|
||
# Citation
|
||
```
|
||
@misc{openthoughts-agent,
|
||
author = {Team, OpenThoughts-Agent},
|
||
title = {{OpenThoughts-Agent: Data Recipes for Agentic Models}},
|
||
howpublished = {https://www.openthoughts.ai/blog/agent},
|
||
year = {2026}
|
||
}
|
||
```
|