Qwen2.5-7B-Agent-AWQ-v1/README.md

base_model, language, license, library_name, pipeline_tag, tags

base_model	language	license	library_name	pipeline_tag	tags
Rarakiyo/Qwen2.5-7B-Agent-v1	en	apache-2.0	transformers	text-generation	awq 4bit agent qwen2.5

Qwen2.5-7B-Agent-AWQ-v1

AWQ 4-bit quantized model from Rarakiyo/Qwen2.5-7B-Agent-v1.

Quantization Config

• Method: AWQ (Activation-aware Weight Quantization)
• Bits: 4
• Group Size: 128
• Version: GEMM
• Zero Point: True

Usage (vLLM)

from vllm import LLM
model = LLM(model="Rarakiyo/Qwen2.5-7B-Agent-AWQ-v1", quantization="awq")