CyberSecQwen-4B-AWQ/README.md

---
license: apache-2.0
base_model: lablab-ai-amd-developer-hackathon/CyberSecQwen-4B
tags:
- qwen3
- cybersecurity
- cti
- cwe-classification
- vulnerability-analysis
- awq
- 4-bit
- quantized
library_name: transformers
pipeline_tag: text-generation
---

# CyberSecQwen-4B-AWQ

4-bit AWQ quantized version of [CyberSecQwen-4B](https://huggingface.co/lablab-ai-amd-developer-hackathon/CyberSecQwen-4B).

## Quantization

| Parameter | Value |
|---|---|
| Method | AWQ (group_size=128, zero_point=True) |
| Weight precision | 4-bit |
| Compute dtype | float16 |
| Calibration samples | 320 CTI-Bench prompts (256 RCM + 64 MCQ, chat-template formatted) |
| Quantization tool | autoawq |
| Calibration hardware | Modal A100 |

## CTI-Bench Evaluation

Evaluated under the [Foundation-Sec-8B protocol](https://arxiv.org/abs/2504.21039):
- Temperature 0.3, max_tokens 512, concurrency 32
- 5 independent trials, zero-shot (no system prompt)
- vLLM v0.20.1 with awq_marlin kernel on Modal L4 GPU

| Task | AWQ 4-bit | GGUF Q4_K_M | FP16 Reference |
|---|---|---|---|
| CTI-MCQ (2,500 items) | **0.5921** ± 0.0083 | 0.5368 ± 0.0048 | 0.5868 ± 0.0029 |
| CTI-RCM (1,000 items) | 0.5814 ± 0.0025 | **0.6254 ± 0.0063** | 0.6664 ± 0.0023 |

**Key findings:**
- **CTI-MCQ**: AWQ 4-bit matches or slightly exceeds FP16 performance (+0.5 points). Better than GGUF Q4_K_M.
- **CTI-RCM**: AWQ 4-bit degrades by 8.5 percentage points vs FP16. GGUF Q4_K_M does better on this task (-4.1 pts).
- AWQ is best for MCQ (general language), GGUF is best for RCM (task-specific classification).

## Trial results

### CTI-MCQ
| Trial | Seed | Accuracy |
|---|---|---|
| 1 | 42 | 0.6016 |
| 2 | 43 | 0.5984 |
| 3 | 44 | 0.5936 |
| 4 | 45 | 0.5780 |
| 5 | 46 | 0.5888 |

### CTI-MCQ

| Trial | Seed | Accuracy |
|---|---|---|
| 1 | 42 | 0.6016 |
| 2 | 43 | 0.5984 |
| 3 | 44 | 0.5936 |
| 4 | 45 | 0.5780 |
| 5 | 46 | 0.5888 |

### CTI-RCM

| Trial | Seed | Accuracy |
|---|---|---|
| 1 | 42 | 0.5790 |
| 2 | 43 | 0.5830 |
| 3 | 44 | 0.5790 |
| 4 | 45 | 0.5840 |
| 5 | 46 | 0.5820 |

## Quantization variants

| Variant | CTI-MCQ | CTI-RCM | Size | Engine |
|---|---|---|---|---|
| [AWQ 4-bit](https://huggingface.co/ree2raz/CyberSecQwen-4B-AWQ) | 0.5921 | 0.5814 | 2.7 GB | vLLM |
| [GGUF Q4_K_M](https://huggingface.co/ree2raz/CyberSecQwen-4B-GGUF) | 0.5368 | 0.6254 | 2.5 GB | llama.cpp |

Choose AWQ for MCQ/general chat, GGUF for vulnerability classification.

## Usage with vLLM

```bash
vllm serve ree2raz/CyberSecQwen-4B-AWQ --quantization awq_marlin --dtype float16
```

## Model Size

| Format | Size |
|---|---|
| Original FP16 | ~8 GB |
| AWQ 4-bit | ~2.7 GB |

## Citation

```bibtex
@misc{{cybersecqwen2026,
  title  = {{CyberSecQwen-4B: A Compact CTI Specialist Fine-Tuned from Qwen3-4B-Instruct-2507 on AMD MI300X}},
  author = {{Mulia, Samuel}},
  year   = {{2026}},
  publisher = {{Hugging Face}},
  url    = {{https://huggingface.co/athena129/CyberSecQwen-4B}}
}}
```

## Evaluation Infrastructure

[GitHub repository](https://github.com/ree2raz/cyberSecQwen_4b_4bit) — Modal scripts for quantization + evaluation.