11 KiB
frameworks, license, tasks, base_model, tags
| frameworks | license | tasks | base_model | tags | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Apache License 2.0 |
|
|
|
MiroThinker-14B-DPO-v0.1-Int8-W8A16 量化高精校准
原模型 okwinds/MiroThinker-14B-DPO-v0.1
📖 关于项目相关的研究，可阅读公众号“觉察流”文章👇
《MiroMind-M1：如何用CAMPO算法打造高效且可复现的全栈开源推理模型》
📖 关于量化损失方面的研究，可阅读公众号“觉察流”文章👇
《Reason Model 的“瘦身计划”：量化技术的得与失》
作者在此 👇🏻 扫一扫
下载
SDK下载
#安装ModelScope
pip install modelscope
#SDK模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('okwinds/MiroThinker-14B-DPO-v0.1-Int8-W8A16')
Git下载
#Git模型下载
git clone https://www.modelscope.cn/okwinds/MiroThinker-14B-DPO-v0.1-Int8-W8A16.git
模型概述
MiroThinker-14B-DPO-v0.1-Int8-W8A16 是一个基于 MiroThinker-14B-DPO-v0.1 的 INT8 量化并校准的模型。
- 模型名称: MiroThinker-14B-DPO-v0.1-Int8-W8A16
- 模型架构: Qwen3ForCausalLM
- 权重量化: INT8
该模型通过将 okwinds/MiroThinker-14B-DPO-v0.1 的权重量化为 INT8 数据类型而实现。
量化过程将每个参数从 16bit 减少到 8bit，将模型占用磁盘空间大小，以及推理时加载模型需要的GPU显存空间，减少到了大约为原模型的1/2。
量化过程中，只有 transformer 中的 Linear 层权重是量化的，其他层均保持为原模型数据类型，采用混合精度的计算方式，尽可能减少模型量化后所带来的精度损失。
在量化过程中，使用了AutoGPTQ，采用 Symmetric group-wise 方式量化。并在此次量化过程中，进行了数据校准，以提升模型生成精度。（相对比 BF16 做到几乎无损）
注意：本模型需要 compute capability > 8.0（Ampere、Ada Lovelace、Hopper 架构）的 Nvidia GPU 来支持 INT8 混合精度计算。
部署推理
推荐使用 vLLM>=0.8.4 ( transformers>=4.51.0 )
Openai api 兼容模式
>>> vllm serve "/home/gavin/llm/MiroThinker-14B-DPO-v0.1-Int8-W8A16" --host 0.0.0.0 --port 8000 --gpu-memory-utilization 0.9 --served-model-name "MiroThinker-14B-DPO-v0.1-Int8-W8A16"
附录 官方 MiroThinker-14B-DPO-v0.1 简介
Introduction
MiroThinker is an open-source agentic model series built on top of Qwen3. Designed for deep research and complex, long-horizon problem solving, it integrates strong capabilities in task decomposition, multi-hop reasoning, retrieval-augmented generation, code execution, web browsing, and document/file processing, making it suitable for a wide range of real-world applications.
We have released the MiroThinker-v0.1 series, including both SFT and DPO variants at parameter scales of 8B, 14B, and 32B. Notably, MiroThinker v0.1 achieves state-of-the-art performance among open-source models on the GAIA benchmark, a rigorous evaluation suite for advanced agentic capabilities, demonstrating its strength in long-context, decision-intensive, and real-world task scenarios.
Online Demo
Welcome to try out our online demo here. In this demo, we have deployed our MiroThinker-32B-DPO-v0.1 along with commercial tools (you can find more details in our GitHub), aiming to deliver a better experience.
Performance
GAIA Benchmark
| Method | Text-103 Best Pass@1 |
Text-103 Pass@1 (Avg@8) |
Val-165 Best Pass@1 |
Val-165 Pass@1 (Avg@8) |
|---|---|---|---|---|
| Search-o1-7B | 17.5 | - | - | - |
| R1-Searcher-7B | 20.4 | - | - | - |
| WebDancer-7B | 31.0 | - | - | - |
| WebSailor-7B | 37.9 | - | - | - |
| CK-Pro-8B | 40.3 | - | 32.7 | - |
| MiroThinker-8B-SFT-v0.1 | 44.7 | 40.1 | 34.6 | 31.8 |
| + Commercial Tools | 46.6 | 42.1 | 37.6 | 33.9 |
| MiroThinker-8B-DPO-v0.1 | 46.6 | 44.8 | 37.0 | 35.4 |
| + Commercial Tools | 50.5 | 46.7 | 38.2 | 35.9 |
| Search-o1-32B | 28.2 | - | - | - |
| WebThinker-32B-RL | 48.5 | - | - | - |
| WebDancer-QwQ-32B | 51.5 | - | - | - |
| WebSailor-32B | 53.2 | - | - | - |
| WebShaper-QwQ-32B | 53.3 | - | - | - |
| WebShaper-72B | 60.1 | - | - | - |
| MiroThinker-14B-SFT-v0.1 | 47.6 | 44.4 | 37.0 | 34.4 |
| + Commercial Tools | 49.5 | 47.5 | 41.8 | 39.8 |
| MiroThinker-14B-DPO-v0.1 | 48.5 | 46.6 | 42.4 | 39.2 |
| + Commercial Tools | 52.4 | 48.5 | 45.5 | 42.0 |
| MiroThinker-32B-SFT-v0.1 | 55.3 | 51.3 | 44.9 | 42.7 |
| + Commercial Tools | 58.3 | 54.2 | 48.5 | 45.8 |
| MiroThinker-32B-DPO-v0.1 | 57.3 | 54.1 | 48.5 | 45.9 |
| + Commercial Tools | 60.2 | 57.9 | 50.9 | 48.9 |
-
Following the practices of WebThinker, WebAgents, and CognitiveKernel, we report the Best Pass@1, the highest score across three runs, which often reflects stronger performance, though it may exhibit some variability. To provide a more stable measure, we additionally report Pass@1 (Avg@8), which offers greater consistency at the cost of slightly lower scores.
-
For consistency with prior open-source works, we evaluate GAIA-Text-103 using the WebAgents LLM-as-judge template, and report results on GAIA-Val-165 using the official GAIA scorer script.
-
By default, we use open-source tools wherever possible, except for the code tool E2B and the Google search tool Serper. We use Whisper, Qwen2.5-VL-72B-Instruct, and Qwen3-235B-A22B-Thinking-2507 in our implementation. The framework can be easily extended to other open-source tools of your choice.
-
Commercial tools were mainly used for multimodal capabilities and certain complex reasoning subtasks. The majority of tasks, including planning, browsing, refinement, navigation, and more, were handled by our models.
More Benchmarks
Coming soon
Quick Start
MiroThinker-v0.1 is trained on our large-scale, high-quality trajectory and preference datasets MiroVerse-v0.1, utilizing the efficient training framework MiroTrain, and enhanced with tool-use capabilities through our agentic framework MiroFlow.
To promote reproducibility and benefit the community, we decided to open-source the entire suite mentioned above. For more technical details, evaluation results, and usage tutorials, please visit our GitHub repository.
License
MiroThinker-v0.1 is licensed under Apache 2.0.
Contact Us
MiroThinker is developed by the MiroMind Foundation Model Team. If you would like to leave us a message, feel free to get in touch. In addition to GitHub, Discord, WeChat, and RedNote, you can also reach us via email at talent@miromind.ai.