174 lines
11 KiB
Markdown
174 lines
11 KiB
Markdown
---
|
||
frameworks:
|
||
- Pytorch
|
||
license: Apache License 2.0
|
||
tasks:
|
||
- text-generation
|
||
base_model:
|
||
- okwinds/MiroThinker-14B-SFT-v0.1
|
||
tags:
|
||
- agent
|
||
- open-source
|
||
- miromind
|
||
---
|
||
|
||
# MiroThinker-14B-DPO-v0.1-Int8-W8A16 量化高精校准
|
||
|
||
原模型 [okwinds/MiroThinker-14B-DPO-v0.1](https://www.modelscope.cn/models/okwinds/MiroThinker-14B-DPO-v0.1)
|
||
|
||
#### 📖 关于项目相关的研究，可阅读公众号“觉察流”文章👇</br>
|
||
|
||
《[MiroMind-M1：如何用CAMPO算法打造高效且可复现的全栈开源推理模型](https://mp.weixin.qq.com/s/REPzzgsUjDMikg4jIo9KRg)》
|
||
|
||
#### 📖 关于量化损失方面的研究，可阅读公众号“觉察流”文章👇</br>
|
||
|
||
《[Reason Model 的“瘦身计划”：量化技术的得与失](https://mp.weixin.qq.com/s/NMGq4UUkfo8GMix5LHnWCg)》
|
||
|
||
---
|
||
|
||
#### _作者在此 👇🏻 扫一扫_
|
||
|
||
<img src="https://www.modelscope.cn/models/okwinds/GPT-2/resolve/master/qrcode_for_jcl_258.jpg" />
|
||
|
||
---
|
||
|
||
## 下载
|
||
|
||
SDK下载
|
||
```bash
|
||
#安装ModelScope
|
||
pip install modelscope
|
||
```
|
||
```python
|
||
#SDK模型下载
|
||
from modelscope import snapshot_download
|
||
model_dir = snapshot_download('okwinds/MiroThinker-14B-DPO-v0.1-Int8-W8A16')
|
||
```
|
||
Git下载
|
||
```
|
||
#Git模型下载
|
||
git clone https://www.modelscope.cn/okwinds/MiroThinker-14B-DPO-v0.1-Int8-W8A16.git
|
||
```
|
||
|
||
## 模型概述
|
||
|
||
MiroThinker-14B-DPO-v0.1-Int8-W8A16 是一个基于 MiroThinker-14B-DPO-v0.1 的 INT8 量化并校准的模型。
|
||
|
||
- **模型名称:** MiroThinker-14B-DPO-v0.1-Int8-W8A16
|
||
- **模型架构:** Qwen3ForCausalLM
|
||
- **权重量化:** INT8
|
||
|
||
该模型通过将 [okwinds/MiroThinker-14B-DPO-v0.1](https://www.modelscope.cn/models/okwinds/MiroThinker-14B-DPO-v0.1) 的权重量化为 INT8 数据类型而实现。
|
||
|
||
量化过程将每个参数从 16bit 减少到 8bit，将模型占用磁盘空间大小，以及推理时加载模型需要的GPU显存空间，减少到了大约为原模型的1/2。
|
||
|
||
量化过程中，只有 transformer 中的 Linear 层权重是量化的，其他层均保持为原模型数据类型，采用混合精度的计算方式，尽可能减少模型量化后所带来的精度损失。
|
||
|
||
在量化过程中，使用了[AutoGPTQ](https://github.com/AutoGPTQ/AutoGPTQ)，采用 Symmetric group-wise 方式量化。并在此次量化过程中，进行了数据校准，以提升模型生成精度。（相对比 BF16 做到几乎无损）
|
||
|
||
> <span style="color: red;">注意：本模型需要 compute capability > 8.0（Ampere、Ada Lovelace、Hopper 架构）的 Nvidia GPU 来支持 INT8 **混合精度**计算。</span>
|
||
|
||
## 部署推理
|
||
|
||
#### 推荐使用 vLLM>=0.8.4 ( transformers>=4.51.0 )
|
||
|
||
Openai api 兼容模式
|
||
|
||
```bash
|
||
>>> vllm serve "/home/gavin/llm/MiroThinker-14B-DPO-v0.1-Int8-W8A16" --host 0.0.0.0 --port 8000 --gpu-memory-utilization 0.9 --served-model-name "MiroThinker-14B-DPO-v0.1-Int8-W8A16"
|
||
```
|
||
|
||
# 附录 官方 MiroThinker-14B-DPO-v0.1 简介
|
||
|
||
<div align="center">
|
||
<img src="https://cdn-uploads.huggingface.co/production/uploads/68525b342230a897a65cc1c0/87mYQ_a-4jpnMkVR4hrgm.png" width="55%" alt="MiroThinker" />
|
||
</div>
|
||
<!-- <hr> -->
|
||
<div align="center">
|
||
|
||
[](https://dr.miromind.ai/)
|
||
[](https://www.modelscope.cn/collections/MiroFlow-c89926d9ab8845)
|
||
[](https://www.modelscope.cn//datasets/okwinds/MiroVerse-v0.1)
|
||
[](https://miromind.ai/blog/miromind-open-deep-research)
|
||
|
||
[](https://github.com/MiroMindAI/MiroThinker)
|
||
[](https://discord.com/invite/GPqEnkzQZd)
|
||
[](https://cdn-uploads.huggingface.co/production/uploads/68525b342230a897a65cc1c0/SGK70isvVpeJwk_fny9sb.png)
|
||
[](https://www.xiaohongshu.com/user/profile/663098830000000003033edc)
|
||
[](https://miromind.ai/)
|
||
|
||
</div>
|
||
|
||
## Introduction
|
||
|
||
MiroThinker is an open-source agentic model series built on top of Qwen3. Designed for deep research and complex, long-horizon problem solving, it integrates strong capabilities in task decomposition, multi-hop reasoning, retrieval-augmented generation, code execution, web browsing, and document/file processing, making it suitable for a wide range of real-world applications.
|
||
|
||
We have released the MiroThinker-v0.1 series, including both SFT and DPO variants at parameter scales of 8B, 14B, and 32B. Notably, MiroThinker v0.1 achieves state-of-the-art performance among open-source models on the [GAIA benchmark](https://huggingface.co/datasets/gaia-benchmark/GAIA), a rigorous evaluation suite for advanced agentic capabilities, demonstrating its strength in long-context, decision-intensive, and real-world task scenarios.
|
||
|
||
## Online Demo
|
||
|
||
Welcome to try out our online demo [here](https://dr.miromind.ai/). In this demo, we have deployed our [MiroThinker-32B-DPO-v0.1](https://huggingface.co/miromind-ai/MiroThinker-32B-DPO-v0.1) along with commercial tools (you can find more details in our [GitHub](https://github.com/MiroMindAI/MiroThinker)), aiming to deliver a better experience.
|
||
|
||
## Performance
|
||
|
||
### GAIA Benchmark
|
||
|
||
| **Method** | Text-103<br>Best Pass@1 | Text-103<br>Pass@1 (Avg@8) | Val-165<br>Best Pass@1 | Val-165<br>Pass@1 (Avg@8) |
|
||
| ----------------------------------------------------------------- | :--: | :--: | :--: | :--: |
|
||
| Search-o1-7B | 17.5 | - | - | - |
|
||
| R1-Searcher-7B | 20.4 | - | - | - |
|
||
| WebDancer-7B | 31.0 | - | - | - |
|
||
| WebSailor-7B | 37.9 | - | - | - |
|
||
| CK-Pro-8B | 40.3 | - | 32.7 | - |
|
||
| MiroThinker-8B-SFT-v0.1 | 44.7 | 40.1 | 34.6 | 31.8 |
|
||
| + Commercial Tools | 46.6 | 42.1 | 37.6 | 33.9 |
|
||
| MiroThinker-8B-DPO-v0.1 | 46.6 | 44.8 | 37.0 | 35.4 |
|
||
| + Commercial Tools | 50.5 | 46.7 | 38.2 | 35.9 |
|
||
| | | | | |
|
||
| Search-o1-32B | 28.2 | - | - | - |
|
||
| WebThinker-32B-RL | 48.5 | - | - | - |
|
||
| WebDancer-QwQ-32B | 51.5 | - | - | - |
|
||
| WebSailor-32B | 53.2 | - | - | - |
|
||
| WebShaper-QwQ-32B | 53.3 | - | - | - |
|
||
| WebShaper-72B | 60.1 | - | - | - |
|
||
| MiroThinker-14B-SFT-v0.1 | 47.6 | 44.4 | 37.0 | 34.4 |
|
||
| + Commercial Tools | 49.5 | 47.5 | 41.8 | 39.8 |
|
||
| MiroThinker-14B-DPO-v0.1 | 48.5 | 46.6 | 42.4 | 39.2 |
|
||
| + Commercial Tools | 52.4 | 48.5 | 45.5 | 42.0 |
|
||
| MiroThinker-32B-SFT-v0.1 | 55.3 | 51.3 | 44.9 | 42.7 |
|
||
| + Commercial Tools | 58.3 | 54.2 | 48.5 | 45.8 |
|
||
| <span style="white-space:nowrap;">MiroThinker-32B-DPO-v0.1</span> | 57.3 | 54.1 | 48.5 | 45.9 |
|
||
| + Commercial Tools | **60.2** | **57.9** | **50.9** | **48.9** |
|
||
|
||
1. Following the practices of WebThinker, WebAgents, and CognitiveKernel, we report the Best Pass@1, the highest score across three runs, which often reflects stronger performance, though it may exhibit some variability. To provide a more stable measure, we additionally report Pass@1 (Avg@8), which offers greater consistency at the cost of slightly lower scores.
|
||
|
||
2. For consistency with prior open-source works, we evaluate GAIA-Text-103 using the WebAgents LLM-as-judge template, and report results on GAIA-Val-165 using the official GAIA scorer script.
|
||
|
||
3. By default, we use open-source tools wherever possible, except for the code tool [E2B](https://github.com/e2b-dev/E2B) and the Google search tool [Serper](https://serper.dev/). We use [Whisper](https://huggingface.co/openai/whisper-large-v3-turbo), [Qwen2.5-VL-72B-Instruct](https://www.modelscope.cn/models/Qwen/Qwen2.5-VL-72B-Instruct), and [Qwen3-235B-A22B-Thinking-2507](https://www.modelscope.cn/models/Qwen/Qwen3-235B-A22B-Thinking-2507) in our implementation. The framework can be easily extended to other open-source tools of your choice.
|
||
|
||
4. Commercial tools were mainly used for multimodal capabilities and certain complex reasoning subtasks. The majority of tasks, including planning, browsing, refinement, navigation, and more, were handled by our models.
|
||
|
||
### More Benchmarks
|
||
|
||
Coming soon
|
||
|
||
## Quick Start
|
||
|
||
MiroThinker-v0.1 is trained on our large-scale, high-quality trajectory and preference datasets [MiroVerse-v0.1](https://www.modelscope.cn/datasets/okwinds/MiroVerse-v0.1), utilizing the efficient training framework [MiroTrain](https://github.com/MiroMindAI/MiroTrain), and enhanced with tool-use capabilities through our agentic framework [MiroFlow](https://github.com/MiroMindAI/MiroFlow).
|
||
|
||
To promote reproducibility and benefit the community, we decided to open-source the entire suite mentioned above. For more technical details, evaluation results, and usage tutorials, please visit our [GitHub repository](https://github.com/MiroMindAI/MiroThinker).
|
||
|
||
## License
|
||
|
||
MiroThinker-v0.1 is licensed under Apache 2.0.
|
||
|
||
## Contact Us
|
||
|
||
MiroThinker is developed by the MiroMind Foundation Model Team.
|
||
If you would like to leave us a message, feel free to get in touch.
|
||
In addition to [GitHub](https://github.com/MiroMindAI/),
|
||
[Discord](https://discord.com/invite/GPqEnkzQZd),
|
||
[WeChat](https://cdn-uploads.huggingface.co/production/uploads/68525b342230a897a65cc1c0/SGK70isvVpeJwk_fny9sb.png),
|
||
and [RedNote](https://www.xiaohongshu.com/user/profile/663098830000000003033edc),
|
||
you can also reach us via email at talent@miromind.ai.
|