---
license: apache-2.0
base_model: Qwen/Qwen3-VL-2B-Instruct
library_name: transformers
---

# Qwen3-VL-2B-Instruct-LLM

The language model of [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct),
extracted as a plain `Qwen3ForCausalLM` checkpoint (vision tower dropped).

The Qwen3-VL-2B text tower is architecturally identical to Qwen3-1.7B dense
(hidden 2048 / 28 layers / 16 Q + 8 KV heads / head_dim 128 / FFN 6144 /
vocab 151936 / tied embeddings). Its interleaved MRoPE assigns text tokens the
same index on all three axes, which reduces exactly to standard 1D RoPE, so
this export keeps the VL `rope_theta` (5e6) and drops `rope_scaling`.

**Equivalence**: text-only logits match the original VL model bit-for-bit in
fp32 (`max |logit diff| = 0.0` on a held-out prompt).

Load with:

```python
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("shushengyang/Qwen3-VL-2B-Instruct-LLM")
```

Produced by `personal/shusheng/mllm/misc/convert_qwen3vl_llm_to_qwen3.py`
(amilabs monorepo); intended as an LLM initialization for ScaleWM MLLM
experiments (vision-language-pretrained LLM vs text-only Qwen3-1.7B).