Qwen3-14B-AWQ-INT4/README.md

license, base_model, tags, language, pipeline_tag

license	base_model	tags	language	pipeline_tag
apache-2.0	Qwen/Qwen3-14B	quantized 4-bit int4 qwen3	en	text-generation

Qwen3-14B-AWQ-INT4

INT4 quantization of Qwen/Qwen3-14B. Built to run on a single consumer GPU (≥10 GB VRAM).

Footprint

Source params	14B
Quantized weights	~9.4 GB on disk
Inference VRAM (incl. KV cache @ 32K context)	~16 GB

Fits any 16 GB+ consumer card. No homelab needed.

Bench

Scored on drawais/needle-1M-bench-mvp (50K-token haystack, real arxiv text):

Metric	Score
Overall recall	90.0%
Paper-anchored	80.0%
Synthetic codes	100.0%

Quick start

vllm serve drawais/Qwen3-14B-AWQ-INT4 --quantization awq_marlin --max-model-len 32768

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("drawais/Qwen3-14B-AWQ-INT4")
model = AutoModelForCausalLM.from_pretrained("drawais/Qwen3-14B-AWQ-INT4", device_map="auto")

Context length

Native: 40,960 tokens (inherits from base model). For longer contexts, enable YaRN rope-scaling per the base model's config — supported by vLLM and Transformers.

License

Apache 2.0 (inherits from base model).