---
library_name: transformers
pipeline_tag: text-generation
base_model:
- Qwen/Qwen3-4B-Instruct-2507
---

# Qwen3-4B-Instruct-2507-GPTQ-INT4
## Model Details

This model is a int4 model with group_size 128 of [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) generated by [vastai modelzoo](https://github.com/Vastai/VastModelZOO). Please follow the license of the original model.

## vllm Inference

- vllm >= v0.11.0
- VVI >= 26.02

```bash
vllm serve vastai-ais/Qwen3-4B-Instruct-2507-GPTQ-INT4 --served_model_name qwen
```

```bash
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d ' {
    "model": "qwen",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Who are you?"}
    ],
    "temperature": 1,
    "max_tokens": 512
  } '
```