---
license: qwen
license_name: Tongyi Qianwen LICENSE AGREEMENT
license_link: LICENSE
pipeline_tag: text-generation
tags:
- QwQ
- gptq
- int8
- 量化修复
- vLLM
- sglang
base_model:
  - Qwen/QwQ-32B
base_model_relation: quantized
---
# 通义千问 QwQ-32B-GPTQ-Int8
基础型 [Qwen/QwQ-32B](https://www.modelscope.cn/models/Qwen/QwQ-32B)

### 【模型更新日期】
``` 
2025-04-24
1. add group 128v3
2. 用黑科技修掉之前 Compute 7 显卡（GPTQ格式）运行时候出现的感叹号问题
3. 删除对话模板的<think>字符，使得模型作答的时候，能完整吐出<think>模块
4. 提供一个无思考模式的对话模板（tokenizer_config-无思考模式.json）供参考把玩。

2025-03-07
1. add group 128
```

### 【依赖】

```
vllm==0.8.4
transformers==4.51.3
```

<div style="
    background: rgba(255, 193, 61, 0.15);
    padding: 16px;
    border-radius: 6px;
    border: 1px solid rgba(255, 165, 0, 0.3);
    margin: 16px 0;
">
### 【💡新版 VLLM 注意事项💡】

#### 1. 需使用V0推理模式
启动vllm之前，先设置环境变量
```
export VLLM_USE_V1=0
```
</div>

### 【模型列表】

| tag      | 文件大小   | 最近更新时间       |
|----------|--------|--------------|
| `g128v3` | `33GB` | `2025-04-24` |
| `g128`   | `33GB` | `2025-03-07` |


### 【模型下载】

```python
from modelscope import snapshot_download
snapshot_download('tclf90/qwq-32b-gptq-int8', cache_dir="本地路径", revision='g128v3')
```

### 【修复内容】

1. 对GPTQ量化的校准做了额外优化；减少模型的 `1.乱吐字`、`2.无限循环`、`3.长文能力丢失`等情况。
2. 支持1卡、2卡及4卡的`tensor-parallel-size`启动。


### 【介绍】
<a href="https://chat.qwenlm.ai/" target="_blank" style="margin: 2px;">
    <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
</a>

## Introduction

QwQ is the reasoning model of the Qwen series. Compared with conventional instruction-tuned models, QwQ, which is capable of thinking and reasoning, can achieve significantly enhanced performance in downstream tasks, especially hard problems. QwQ-32B is the medium-sized reasoning model, which is capable of achieving competitive performance against state-of-the-art reasoning models, e.g., DeepSeek-R1, o1-mini.

**This repo contains the QwQ 32B model**, which has the following features:
- Type: Causal Language Models
- Training Stage: Pretraining & Post-training (Supervised Finetuning and Reinforcement Learning)
- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
- Number of Parameters: 32.5B
- Number of Paramaters (Non-Embedding): 31.0B
- Number of Layers: 64
- Number of Attention Heads (GQA): 40 for Q and 8 for KV
- Context Length: Full 131,072 tokens

**Note:** For the best experience, please review the [usage guidelines](#usage-guidelines) before deploying QwQ models.

For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5/), [GitHub](https://github.com/QwenLM/Qwen2.5), and [Documentation](https://qwen.readthedocs.io/en/latest/).