59 lines
2.6 KiB
Markdown
59 lines
2.6 KiB
Markdown
---
|
|
license: apache-2.0
|
|
license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/blob/main/LICENSE
|
|
language:
|
|
- en
|
|
base_model:
|
|
- Qwen/Qwen2.5-Coder-7B-Instruct
|
|
pipeline_tag: text-generation
|
|
library_name: transformers
|
|
tags:
|
|
- code
|
|
- codeqwen
|
|
- chat
|
|
- qwen
|
|
- qwen-coder
|
|
---
|
|
|
|
|
|
# Qwen2.5-Coder-7B-Instruct-AWQ
|
|
|
|
## Introduction
|
|
|
|
Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
|
|
|
|
- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.
|
|
- A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
|
|
- **Long-context Support** up to 128K tokens.
|
|
|
|
**This repo contains the AWQ-quantized 4-bit instruction-tuned 7B Qwen2.5-Coder model**, which has the following features:
|
|
- Type: Causal Language Models
|
|
- Training Stage: Pretraining & Post-training
|
|
- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
|
|
- Number of Parameters: 7.61B
|
|
- Number of Paramaters (Non-Embedding): 6.53B
|
|
- Number of Layers: 28
|
|
- Number of Attention Heads (GQA): 28 for Q and 4 for KV
|
|
- Context Length: Full 131,072 tokens
|
|
- Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy Qwen2.5 for handling long texts.
|
|
- Quantization: AWQ 4-bit
|
|
|
|
For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186).
|
|
|
|
## Requirements
|
|
|
|
The code of Qwen2.5-Coder has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`.
|
|
|
|
With `transformers<4.37.0`, you will encounter the following error:
|
|
```
|
|
KeyError: 'qwen2'
|
|
```
|
|
|
|
This model depends on `gptqmodel`. Please pin the version to `7.3.1` (newer versions are currently incompatible):
|
|
```
|
|
pip install gptqmodel==7.3.1
|
|
```
|
|
|
|
Also check out our [AWQ documentation](https://qwen.readthedocs.io/en/latest/quantization/awq.html) for more usage guide.
|
|
|