MusePublic/Qwen-image
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: MusePublic/Qwen-image
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-04-10 11:24:55 +08:00
commit 98d01e2546
31 changed files with 306577 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

61
.gitattributes vendored Normal file
View File

@@ -0,0 +1,61 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.gguf* filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
text_encoder/model-00001-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
text_encoder/model-00002-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
text_encoder/model-00003-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
text_encoder/model-00004-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
vae/diffusion_pytorch_model.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00007-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00009-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00001-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00002-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00004-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00008-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00003-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00005-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text
transformer/diffusion_pytorch_model-00006-of-00009.safetensors filter=lfs diff=lfs merge=lfs -text

167
README.md Normal file
View File

@@ -0,0 +1,167 @@
---
base_model:
- Qwen/Qwen-Image
base_model_relation: repackage
frameworks:
- Pytorch
license: Apache License 2.0
tags:
- ckpt
tasks:
- text-to-image-synthesis
vision_foundation: QWEN_IMAGE_20_B
#model-type:
##如 gpt、phi、llama、chatglm、baichuan 等
#- gpt
#domain:
##如 nlp、cv、audio、multi-modal
#- nlp
#language:
##语言代码列表 https://help.aliyun.com/document_detail/215387.html?spm=a2c4g.11186623.0.0.9f8d7467kni6Aa
#- cn
#metrics:
##如 CIDEr、Blue、ROUGE 等
#- CIDEr
#tags:
##各种自定义，包括 pretrained、fine-tuned、instruction-tuned、RL-tuned 等训练方法和其他
#- pretrained
#tools:
##如 vllm、fastchat、llamacpp、AdaSeq 等
#- vllm
---
<p align="center">
<img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/qwen_image_logo.png" width="400"/>
<p>
<p align="center">
💜 <a href="https://chat.qwen.ai/"><b>Qwen Chat</b></a>&nbsp&nbsp | &nbsp&nbsp🤗 <a href="https://huggingface.co/Qwen">Hugging Face</a>&nbsp&nbsp | &nbsp&nbsp🤖 <a href="https://modelscope.cn/organization/qwen">ModelScope</a>&nbsp&nbsp | &nbsp&nbsp 📑 <a href="https://arxiv.org/abs/xxx">Arxiv</a> &nbsp&nbsp | &nbsp&nbsp 📑 <a href="https://qwenlm.github.io/blog/qwen-image/">Blog</a> &nbsp&nbsp
<br>
🖥️ <a href="https://huggingface.co/spaces/Qwen/Qwen-Image-Demo">Demo</a>&nbsp&nbsp | &nbsp&nbsp💬 <a href="https://github.com/QwenLM/Qwen/blob/main/assets/wechat.png">WeChat (微信)</a>&nbsp&nbsp | &nbsp&nbsp🫨 <a href="https://discord.gg/CV4E9rpNSD">Discord</a>&nbsp&nbsp
</p>
<p align="center">
<img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/meet_en.png#center" width="800"/>
<p>
## Introduction
We are thrilled to release **Qwen-Image**, an image generation foundation model in the Qwen series that achieves significant advances in **complex text rendering** and **precise image editing**. Experiments show strong general capabilities in both image generation and editing, with exceptional performance in text rendering, especially for Chinese.
![](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/bench.png#center)
## News
- 2025.08.04: We released [Qwen-Image Tech Report]().
- 2025.08.04: We released Qwen-Image weights! Check at [huggingface](https://huggingface.co/Qwen/Qwen-Image) and [Modelscope](https://modelscope.cn/models/Qwen/Qwen-Image)!
- 2025.08.04: We released Qwen-Image! Check our [blog](https://qwenlm.github.io/blog/qwen-image) for more details!
## Quick Start
Install the latest version of diffusers
```
pip install git+https://github.com/huggingface/diffusers
```
The following contains a code snippet illustrating how to use the model to generate images based on text prompts:
```python
from diffusers import DiffusionPipeline
import torch
model_name = "Qwen/Qwen-Image"
# Load the pipeline
if torch.cuda.is_available():
torch_dtype = torch.bfloat16
device = "cuda"
else:
torch_dtype = torch.float32
device = "cpu"
pipe = DiffusionPipeline.from_pretrained(model_name, torch_dtype=torch_dtype)
pipe = pipe.to(device)
positive_magic = "Ultra HD, 4K, cinematic composition." # for english prompt
# positive_magic = "超清，4K，电影级构图" # for chinese prompt
# Generate image
prompt = '''A coffee shop entrance features a chalkboard sign reading "Qwen Coffee 😊 $2 per cup," with a neon light beside it displaying "通义千问". Next to it hangs a poster showing a beautiful Chinese woman, and beneath the poster is written "π≈3.1415926-53589793-23846264-33832795-02384197". Ultra HD, 4K, cinematic composition'''
negative_prompt = " "
# Generate with different aspect ratios
aspect_ratios = {
"1:1": (1328, 1328),
"16:9": (1664, 928),
"9:16": (928, 1664),
"4:3": (1472, 1140),
"3:4": (1140, 1472)
}
width, height = aspect_ratios["16:9"]
image = pipe(
prompt=prompt + positive_magic,
width=width,
height=height,
num_inference_steps=50,
true_cfg_scale=4.0,
generator=torch.Generator(device="cuda").manual_seed(42)
).images[0]
image.save("example.png")
```
## Show Cases
One of its standout capabilities is high-fidelity text rendering across diverse images. Whether it’s alphabetic languages like English or logographic scripts like Chinese, Qwen-Image preserves typographic details, layout coherence, and contextual harmony with stunning accuracy. Text isn’t just overlaid—it’s seamlessly integrated into the visual fabric.
![](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/s1.jpg#center)
Beyond text, Qwen-Image excels at general image generation with support for a wide range of artistic styles. From photorealistic scenes to impressionist paintings, from anime aesthetics to minimalist design, the model adapts fluidly to creative prompts, making it a versatile tool for artists, designers, and storytellers.
![](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/s2.jpg#center)
When it comes to image editing, Qwen-Image goes far beyond simple adjustments. It enables advanced operations such as style transfer, object insertion or removal, detail enhancement, text editing within images, and even human pose manipulation—all with intuitive input and coherent output. This level of control brings professional-grade editing within reach of everyday users.
![](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/s3.jpg#center)
But Qwen-Image doesn’t just create or edit—it understands. It supports a suite of image understanding tasks, including object detection, semantic segmentation, depth and edge (Canny) estimation, novel view synthesis, and super-resolution. These capabilities, while technically distinct, can all be seen as specialized forms of intelligent image editing, powered by deep visual comprehension.
![](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/s4.jpg#center)
Together, these features make Qwen-Image not just a tool for generating pretty pictures, but a comprehensive foundation model for intelligent visual creation and manipulation—where language, layout, and imagery converge.
## License Agreement
Qwen-Image is licensed under Apache 2.0.
## Citation
We kindly encourage citation of our work if you find it useful.
```bibtex
@article{qwen-image,
title={Qwen-Image Technical Report},
author={Qwen Team},
journal={arXiv preprint},
year={2025}
}
```
## Contact and Join Us
If you'd like to get in touch with our research team, we’d love to hear from you! Join our [Discord](https://discord.gg/z3GAxXZ9Ce) or scan the QR code to connect via our [WeChat groups](assets/wechat.png) — we’re always open to discussion and collaboration.
If you have questions about this repository, feedback to share, or want to contribute directly, we welcome your issues and pull requests on GitHub. Your contributions help make Qwen-Image better for everyone.
If you’re passionate about fundamental research, we’re hiring full-time employees (FTEs) and research interns. Don’t wait — reach out to us at fulai.hr@alibaba-inc.com

16
configuration.json Normal file
View File

@@ -0,0 +1,16 @@
{
"aigc_model": true,
"framework": "Pytorch",
"task": "text-to-image-synthesis",
"model_file_location": [
"transformer/diffusion_pytorch_model-00001-of-00009.safetensors",
"transformer/diffusion_pytorch_model-00002-of-00009.safetensors",
"transformer/diffusion_pytorch_model-00003-of-00009.safetensors",
"transformer/diffusion_pytorch_model-00004-of-00009.safetensors",
"transformer/diffusion_pytorch_model-00005-of-00009.safetensors",
"transformer/diffusion_pytorch_model-00006-of-00009.safetensors",
"transformer/diffusion_pytorch_model-00007-of-00009.safetensors",
"transformer/diffusion_pytorch_model-00008-of-00009.safetensors",
"transformer/diffusion_pytorch_model-00009-of-00009.safetensors"
]
}

24
model_index.json Normal file
View File

@@ -0,0 +1,24 @@
{
"_class_name": "QwenImagePipeline",
"_diffusers_version": "0.34.0.dev0",
"scheduler": [
"diffusers",
"FlowMatchEulerDiscreteScheduler"
],
"text_encoder": [
"transformers",
"Qwen2_5_VLForConditionalGeneration"
],
"tokenizer": [
"transformers",
"Qwen2Tokenizer"
],
"transformer": [
"diffusers",
"QwenImageTransformer2DModel"
],
"vae": [
"diffusers",
"AutoencoderKLQwenImage"
]
}

18
scheduler/scheduler_config.json Normal file
View File

@@ -0,0 +1,18 @@
{
"_class_name": "FlowMatchEulerDiscreteScheduler",
"_diffusers_version": "0.34.0.dev0",
"base_image_seq_len": 256,
"base_shift": 0.5,
"invert_sigmas": false,
"max_image_seq_len": 4096,
"max_shift": 1.15,
"num_train_timesteps": 1000,
"shift": 1.0,
"shift_terminal": null,
"stochastic_sampling": false,
"time_shift_type": "exponential",
"use_beta_sigmas": false,
"use_dynamic_shifting": true,
"use_exponential_sigmas": false,
"use_karras_sigmas": false
}

135
text_encoder/config.json Normal file
View File

@@ -0,0 +1,135 @@
{
"architectures": [
"Qwen2_5_VLForConditionalGeneration"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 3584,
"image_token_id": 151655,
"initializer_range": 0.02,
"intermediate_size": 18944,
"max_position_embeddings": 128000,
"max_window_layers": 28,
"model_type": "qwen2_5_vl",
"num_attention_heads": 28,
"num_hidden_layers": 28,
"num_key_value_heads": 4,
"rms_norm_eps": 1e-06,
"rope_scaling": {
"mrope_section": [
16,
24,
24
],
"rope_type": "default",
"type": "default"
},
"rope_theta": 1000000.0,
"sliding_window": 32768,
"text_config": {
"architectures": [
"Qwen2_5_VLForConditionalGeneration"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 3584,
"image_token_id": null,
"initializer_range": 0.02,
"intermediate_size": 18944,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"max_position_embeddings": 128000,
"max_window_layers": 28,
"model_type": "qwen2_5_vl_text",
"num_attention_heads": 28,
"num_hidden_layers": 28,
"num_key_value_heads": 4,
"rms_norm_eps": 1e-06,
"rope_scaling": {
"mrope_section": [
16,
24,
24
],
"rope_type": "default",
"type": "default"
},
"rope_theta": 1000000.0,
"sliding_window": null,
"torch_dtype": "float32",
"use_cache": true,
"use_sliding_window": false,
"video_token_id": null,
"vision_end_token_id": 151653,
"vision_start_token_id": 151652,
"vision_token_id": 151654,
"vocab_size": 152064
},
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.53.1",
"use_cache": true,
"use_sliding_window": false,
"video_token_id": 151656,
"vision_config": {
"depth": 32,
"fullatt_block_indexes": [
7,
15,
23,
31
],
"hidden_act": "silu",
"hidden_size": 1280,
"in_channels": 3,
"in_chans": 3,
"initializer_range": 0.02,
"intermediate_size": 3420,
"model_type": "qwen2_5_vl",
"num_heads": 16,
"out_hidden_size": 3584,
"patch_size": 14,
"spatial_merge_size": 2,
"spatial_patch_size": 14,
"temporal_patch_size": 2,
"tokens_per_second": 2,
"torch_dtype": "float32",
"window_size": 112
},
"vision_end_token_id": 151653,
"vision_start_token_id": 151652,
"vision_token_id": 151654,
"vocab_size": 152064
}

14
text_encoder/generation_config.json Normal file
View File

@@ -0,0 +1,14 @@
{
"bos_token_id": 151643,
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"repetition_penalty": 1.05,
"temperature": 0.1,
"top_k": 1,
"top_p": 0.001,
"transformers_version": "4.53.1"
}

3
text_encoder/model-00001-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d725335e4ea2399be706469e4b8807716a8fa64bd03468252e9f7acf2415fee4
size 4968243304

3
text_encoder/model-00002-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b1830db6908dcc76df3a71492acbcf2b8cac130114cf1f3c2d9edae8de8c6de3
size 4991495816

3
text_encoder/model-00003-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:09c1807c6d00d7cab94f7db39d4c02ebb8537225ccde383861ac48db97945aa6
size 4932751040

3
text_encoder/model-00004-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5dd068336d14d45ffb43cef374d286cc6ba9d8741b028f90a7d040d847961f4a
size 1691924384

737
text_encoder/model.safetensors.index.json Normal file
View File

@@ -0,0 +1,737 @@
{
"metadata": {
"total_parameters": 8292166656,
"total_size": 16584333312
},
"weight_map": {
"lm_head.weight": "model-00004-of-00004.safetensors",
"model.embed_tokens.weight": "model-00001-of-00004.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.16.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.17.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.17.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.17.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.18.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.18.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.26.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.27.self_attn.k_proj.bias": "model-00004-of-00004.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.27.self_attn.q_proj.bias": "model-00004-of-00004.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.27.self_attn.v_proj.bias": "model-00004-of-00004.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.5.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.6.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.6.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.norm.weight": "model-00004-of-00004.safetensors",
"visual.blocks.0.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.0.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.0.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.0.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.0.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.0.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.0.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.0.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.0.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.1.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.1.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.1.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.1.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.1.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.1.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.1.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.1.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.1.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.10.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.10.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.10.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.10.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.10.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.10.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.10.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.10.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.10.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.10.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.10.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.10.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.11.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.11.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.11.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.11.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.11.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.11.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.11.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.11.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.11.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.11.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.11.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.11.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.12.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.12.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.12.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.12.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.12.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.12.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.12.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.12.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.12.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.12.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.12.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.12.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.13.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.13.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.13.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.13.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.13.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.13.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.13.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.13.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.13.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.13.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.13.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.13.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.14.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.14.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.14.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.14.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.14.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.14.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.14.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.14.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.14.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.14.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.14.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.14.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.15.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.15.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.15.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.15.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.15.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.15.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.15.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.15.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.15.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.15.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.15.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.15.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.16.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.16.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.16.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.16.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.16.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.16.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.16.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.16.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.16.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.16.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.16.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.16.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.17.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.17.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.17.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.17.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.17.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.17.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.17.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.17.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.17.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.17.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.17.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.17.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.18.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.18.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.18.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.18.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.18.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.18.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.18.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.18.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.18.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.18.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.18.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.18.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.19.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.19.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.19.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.19.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.19.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.19.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.19.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.19.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.19.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.19.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.19.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.19.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.2.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.2.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.2.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.2.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.2.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.2.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.2.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.2.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.2.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.20.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.20.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.20.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.20.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.20.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.20.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.20.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.20.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.20.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.20.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.20.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.20.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.21.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.21.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.21.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.21.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.21.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.21.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.21.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.21.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.21.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.21.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.21.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.21.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.22.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.22.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.22.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.22.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.22.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.22.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.22.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.22.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.22.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.22.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.22.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.22.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.23.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.23.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.23.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.23.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.23.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.23.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.23.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.23.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.23.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.23.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.23.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.23.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.24.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.24.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.24.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.24.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.24.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.24.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.24.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.24.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.24.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.24.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.24.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.24.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.25.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.25.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.25.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.25.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.25.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.25.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.25.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.25.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.25.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.25.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.25.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.25.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.26.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.26.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.26.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.26.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.26.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.26.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.26.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.26.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.26.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.26.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.26.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.26.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.27.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.27.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.27.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.27.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.27.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.27.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.27.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.27.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.27.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.27.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.27.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.27.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.28.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.28.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.28.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.28.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.28.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.28.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.28.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.28.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.28.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.28.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.28.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.28.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.29.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.29.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.29.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.29.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.29.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.29.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.29.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.29.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.29.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.29.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.29.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.29.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.3.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.3.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.3.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.3.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.3.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.3.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.3.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.3.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.3.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.30.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.30.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.30.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.30.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.30.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.30.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.30.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.30.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.30.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.30.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.30.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.30.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.31.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.31.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.31.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.31.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.31.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.31.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.31.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.31.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.31.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.31.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.31.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.31.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.4.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.4.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.4.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.4.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.4.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.4.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.4.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.4.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.4.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.5.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.5.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.5.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.5.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.5.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.5.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.5.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.5.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.5.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.6.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.6.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.6.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.6.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.6.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.6.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.6.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.6.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.6.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.7.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.7.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.7.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.7.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.7.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.7.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.7.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.7.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.7.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.8.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.8.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.8.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.8.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.8.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.8.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.8.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.8.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.8.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.8.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.8.norm2.weight": "model-00001-of-00004.safetensors",
"visual.blocks.9.attn.proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.9.attn.proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.9.attn.qkv.bias": "model-00001-of-00004.safetensors",
"visual.blocks.9.attn.qkv.weight": "model-00001-of-00004.safetensors",
"visual.blocks.9.mlp.down_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.9.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.9.mlp.gate_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.9.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.9.mlp.up_proj.bias": "model-00001-of-00004.safetensors",
"visual.blocks.9.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"visual.blocks.9.norm1.weight": "model-00001-of-00004.safetensors",
"visual.blocks.9.norm2.weight": "model-00001-of-00004.safetensors",
"visual.merger.ln_q.weight": "model-00001-of-00004.safetensors",
"visual.merger.mlp.0.bias": "model-00001-of-00004.safetensors",
"visual.merger.mlp.0.weight": "model-00001-of-00004.safetensors",
"visual.merger.mlp.2.bias": "model-00001-of-00004.safetensors",
"visual.merger.mlp.2.weight": "model-00001-of-00004.safetensors",
"visual.patch_embed.proj.weight": "model-00001-of-00004.safetensors"
}
}

24
tokenizer/added_tokens.json Normal file
View File

@@ -0,0 +1,24 @@
{
"</tool_call>": 151658,
"<tool_call>": 151657,
"<|box_end|>": 151649,
"<|box_start|>": 151648,
"<|endoftext|>": 151643,
"<|file_sep|>": 151664,
"<|fim_middle|>": 151660,
"<|fim_pad|>": 151662,
"<|fim_prefix|>": 151659,
"<|fim_suffix|>": 151661,
"<|im_end|>": 151645,
"<|im_start|>": 151644,
"<|image_pad|>": 151655,
"<|object_ref_end|>": 151647,
"<|object_ref_start|>": 151646,
"<|quad_end|>": 151651,
"<|quad_start|>": 151650,
"<|repo_name|>": 151663,
"<|video_pad|>": 151656,
"<|vision_end|>": 151653,
"<|vision_pad|>": 151654,
"<|vision_start|>": 151652
}

54
tokenizer/chat_template.jinja Normal file
View File

@@ -0,0 +1,54 @@
{%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages[0]['role'] == 'system' %}
{{- messages[0]['content'] }}
{%- else %}
{{- 'You are a helpful assistant.' }}
{%- endif %}
{{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages[0]['role'] == 'system' %}
{{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
{%- else %}
{{- '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- for message in messages %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
{%- elif message.role == "assistant" %}
{{- '<|im_start|>' + message.role }}
{%- if message.content %}
{{- '\n' + message.content }}
{%- endif %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.function is defined %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '\n<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{{- tool_call.arguments | tojson }}
{{- '}\n</tool_call>' }}
{%- endfor %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- message.content }}
{{- '\n</tool_response>' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- endif %}

151388
tokenizer/merges.txt Normal file
View File

File diff suppressed because it is too large Load Diff

31
tokenizer/special_tokens_map.json Normal file
View File

@@ -0,0 +1,31 @@
{
"additional_special_tokens": [
"<|im_start|>",
"<|im_end|>",
"<|object_ref_start|>",
"<|object_ref_end|>",
"<|box_start|>",
"<|box_end|>",
"<|quad_start|>",
"<|quad_end|>",
"<|vision_start|>",
"<|vision_end|>",
"<|vision_pad|>",
"<|image_pad|>",
"<|video_pad|>"
],
"eos_token": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

207
tokenizer/tokenizer_config.json Normal file
View File

@@ -0,0 +1,207 @@
{
"add_bos_token": false,
"add_prefix_space": false,
"added_tokens_decoder": {
"151643": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151644": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151645": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151646": {
"content": "<|object_ref_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151647": {
"content": "<|object_ref_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151648": {
"content": "<|box_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151649": {
"content": "<|box_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151650": {
"content": "<|quad_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151651": {
"content": "<|quad_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151652": {
"content": "<|vision_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151653": {
"content": "<|vision_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151654": {
"content": "<|vision_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151655": {
"content": "<|image_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151656": {
"content": "<|video_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151657": {
"content": "<tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151658": {
"content": "</tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151659": {
"content": "<|fim_prefix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151660": {
"content": "<|fim_middle|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151661": {
"content": "<|fim_suffix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151662": {
"content": "<|fim_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151663": {
"content": "<|repo_name|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151664": {
"content": "<|file_sep|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
}
},
"additional_special_tokens": [
"<|im_start|>",
"<|im_end|>",
"<|object_ref_start|>",
"<|object_ref_end|>",
"<|box_start|>",
"<|box_end|>",
"<|quad_start|>",
"<|quad_end|>",
"<|vision_start|>",
"<|vision_end|>",
"<|vision_pad|>",
"<|image_pad|>",
"<|video_pad|>"
],
"bos_token": null,
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"extra_special_tokens": {},
"model_max_length": 131072,
"pad_token": "<|endoftext|>",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}

151645
tokenizer/vocab.json Normal file
View File

File diff suppressed because it is too large Load Diff

18
transformer/config.json Normal file
View File

@@ -0,0 +1,18 @@
{
"_class_name": "QwenImageTransformer2DModel",
"_diffusers_version": "0.34.0.dev0",
"attention_head_dim": 128,
"axes_dims_rope": [
16,
56,
56
],
"guidance_embeds": false,
"in_channels": 64,
"joint_attention_dim": 3584,
"num_attention_heads": 24,
"num_layers": 60,
"out_channels": 16,
"patch_size": 2,
"pooled_projection_dim": 768
}

3
transformer/diffusion_pytorch_model-00001-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9f33a59093af3abcc2836d4cf4b7bd122c238ca70a26c70f34fdde64646b3bcd
size 4989364312

3
transformer/diffusion_pytorch_model-00002-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:49d580e85b932e15c226e68a9d5549629878d3a128da7662d28401956cee0238
size 4984214160

3
transformer/diffusion_pytorch_model-00003-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0d72f0cbd0d6c2bdc2e1e73e41ac143452fc9e4570d1362203498832302de69f
size 4946470000

3
transformer/diffusion_pytorch_model-00004-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ee87991bfa1bc8e7553034b07099151532524101ca1581c4f3a86292814e9f0d
size 4984213736

3
transformer/diffusion_pytorch_model-00005-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:83291a65be26fe688db4cf81383749831b623a6fcb1a71c851bafb55ddc0058d
size 4946471896

3
transformer/diffusion_pytorch_model-00006-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:61e936a995e7f4ce08d44b9d6ffb56dc65dd61200288876ee2fda72da322583d
size 4946451560

3
transformer/diffusion_pytorch_model-00007-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:854bc9e4a65c11c256f2fef31606a6407e9a425e8bde394db8be71416c515ead
size 4908690520

3
transformer/diffusion_pytorch_model-00008-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:caedc7cc2914ab113cfbb3684cf072350201182ae8de1a7308e419385987ae40
size 4984232856

3
transformer/diffusion_pytorch_model-00009-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae0fef149890f61d54572baa1e806de97156c7ec5b7bbbbbbf27a2d87f33af61
size 1170918840

1940
transformer/diffusion_pytorch_model.safetensors.index.json Normal file
View File

File diff suppressed because it is too large Load Diff

56
vae/config.json Normal file
View File

@@ -0,0 +1,56 @@
{
"_class_name": "AutoencoderKLQwenImage",
"_diffusers_version": "0.34.0.dev0",
"attn_scales": [],
"base_dim": 96,
"dim_mult": [
1,
2,
4,
4
],
"dropout": 0.0,
"latents_mean": [
-0.7571,
-0.7089,
-0.9113,
0.1075,
-0.1745,
0.9653,
-0.1517,
1.5508,
0.4134,
-0.0715,
0.5517,
-0.3632,
-0.1922,
-0.9497,
0.2503,
-0.2921
],
"latents_std": [
2.8184,
1.4541,
2.3275,
2.6558,
1.2196,
1.7708,
2.6052,
2.0743,
3.2687,
2.1526,
2.8652,
1.5579,
1.6382,
1.1253,
2.8251,
1.916
],
"num_res_blocks": 2,
"temperal_downsample": [
false,
true,
true
],
"z_dim": 16
}

3
vae/diffusion_pytorch_model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0c8bc8b758c649abef9ea407b95408389a3b2f610d0d10fcb054fe171d0a8344
size 253806966