---
license: qwen
license_name: Tongyi Qianwen LICENSE AGREEMENT
license_link: LICENSE
pipeline_tag: text-generation
tags:
- qwen2
- gptq
- int8
- 量化修复
- vLLM
---

# 通义千问2-14B-Merge-GPTQ-Int8-量化修复
HF原模型 [paperplanedeemo/Qwen2-14B-merge](https://huggingface.co/paperplanedeemo/Qwen2-14B-merge)


### 【模型更新日期】
``` 2024-06-13 ```

### 【模型大小】
`13.3GB`

### 【修复内容】

1. 对GPTQ量化的校准做了额外优化；减少量化模型的 `1.乱吐字`、`2.无限循环`、`3.长文能力丢失`等情况。

### 【更新日志】

```
2024-06-13
首次commit
```

### 【介绍】
<i>注意：非官方模型</i>

Qwen2-14B-merge is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
```yaml
dtype: float16
merge_method: passthrough
slices:
  - sources:
      - layer_range: [0, 6]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [3, 9]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [6, 12]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [9, 15]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [12, 18]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [15, 21]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [18, 24]
        model: Qwen/Qwen2-7B-Instruct
  - sources:
      - layer_range: [21, 28]
        model: Qwen/Qwen2-7B-Instruct
```

[更多详情...](https://huggingface.co/paperplanedeemo/Qwen2-14B-merge)

### 【同期量化修复模型】
待工作完成后补充...

### 【模型下载】
```python
from modelscope import snapshot_download
model_dir = snapshot_download('tclf90/模型名', cache_dir="本地路径")
```

### 【[vLLM](https://github.com/vllm-project/vllm)推理（目前仅限Linux）】
#### 1. Python 简易调试

待工作完成后补充...

#### 2. 类ChatGPT RESTFul API Server
```
>>> python -m vllm.entrypoints.openai.api_server --model 本地路径/tclf90/模型名称
```