初始化项目,由ModelHub XC社区提供模型
Model: ShushengYang/Qwen3-VL-2B-Instruct-LLM Source: Original Platform
This commit is contained in:
30
README.md
Normal file
30
README.md
Normal file
@@ -0,0 +1,30 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: Qwen/Qwen3-VL-2B-Instruct
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
# Qwen3-VL-2B-Instruct-LLM
|
||||
|
||||
The language model of [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct),
|
||||
extracted as a plain `Qwen3ForCausalLM` checkpoint (vision tower dropped).
|
||||
|
||||
The Qwen3-VL-2B text tower is architecturally identical to Qwen3-1.7B dense
|
||||
(hidden 2048 / 28 layers / 16 Q + 8 KV heads / head_dim 128 / FFN 6144 /
|
||||
vocab 151936 / tied embeddings). Its interleaved MRoPE assigns text tokens the
|
||||
same index on all three axes, which reduces exactly to standard 1D RoPE, so
|
||||
this export keeps the VL `rope_theta` (5e6) and drops `rope_scaling`.
|
||||
|
||||
**Equivalence**: text-only logits match the original VL model bit-for-bit in
|
||||
fp32 (`max |logit diff| = 0.0` on a held-out prompt).
|
||||
|
||||
Load with:
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM
|
||||
model = AutoModelForCausalLM.from_pretrained("shushengyang/Qwen3-VL-2B-Instruct-LLM")
|
||||
```
|
||||
|
||||
Produced by `personal/shusheng/mllm/misc/convert_qwen3vl_llm_to_qwen3.py`
|
||||
(amilabs monorepo); intended as an LLM initialization for ScaleWM MLLM
|
||||
experiments (vision-language-pretrained LLM vs text-only Qwen3-1.7B).
|
||||
Reference in New Issue
Block a user