31 lines
557 B
Markdown
31 lines
557 B
Markdown
|
---
|
||
|
|
base_model: Rarakiyo/Qwen2.5-7B-Agent-v1
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
license: apache-2.0
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- awq
|
||
|
|
- 4bit
|
||
|
|
- agent
|
||
|
|
- qwen2.5
|
||
|
|
---
|
||
|
|
|
||
|
|
# Qwen2.5-7B-Agent-AWQ-v1
|
||
|
|
|
||
|
|
AWQ 4-bit quantized model from **Rarakiyo/Qwen2.5-7B-Agent-v1**.
|
||
|
|
|
||
|
|
## Quantization Config
|
||
|
|
- **Method**: AWQ (Activation-aware Weight Quantization)
|
||
|
|
- **Bits**: 4
|
||
|
|
- **Group Size**: 128
|
||
|
|
- **Version**: GEMM
|
||
|
|
- **Zero Point**: True
|
||
|
|
|
||
|
|
## Usage (vLLM)
|
||
|
|
```python
|
||
|
|
from vllm import LLM
|
||
|
|
model = LLM(model="Rarakiyo/Qwen2.5-7B-Agent-AWQ-v1", quantization="awq")
|
||
|
|
```
|