初始化项目，由ModelHub XC社区提供模型
Model: remyxai/SpaceThinker-Qwen2.5VL-3B Source: Original Platform
This commit is contained in:
ModelHub XC
42
.gitattributes
vendored
Normal file
42
.gitattributes
vendored
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
|
images/example_2.png filter=lfs diff=lfs merge=lfs -text
|
||||||
|
images/example_3.png filter=lfs diff=lfs merge=lfs -text
|
||||||
|
gguf/spacethinker-qwen2.5VL-3B-F16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
gguf/spacethinker-qwen2.5vl-3b-vision.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
assets/SpaceLLaVA.pdf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
52
LICENSE
Normal file
52
LICENSE
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
Qwen RESEARCH LICENSE AGREEMENT
|
||||||
|
|
||||||
|
Qwen RESEARCH LICENSE AGREEMENT Release Date: September 19, 2024
|
||||||
|
|
||||||
|
By clicking to agree or by using or distributing any portion or element of the Qwen Materials, you will be deemed to have recognized and accepted the content of this Agreement, which is effective immediately.
|
||||||
|
|
||||||
|
1. Definitions
|
||||||
|
a. This Qwen RESEARCH LICENSE AGREEMENT (this "Agreement") shall mean the terms and conditions for use, reproduction, distribution and modification of the Materials as defined by this Agreement.
|
||||||
|
b. "We" (or "Us") shall mean Alibaba Cloud.
|
||||||
|
c. "You" (or "Your") shall mean a natural person or legal entity exercising the rights granted by this Agreement and/or using the Materials for any purpose and in any field of use.
|
||||||
|
d. "Third Parties" shall mean individuals or legal entities that are not under common control with us or you.
|
||||||
|
e. "Qwen" shall mean the large language models, and software and algorithms, consisting of trained model weights, parameters (including optimizer states), machine-learning model code, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing distributed by us.
|
||||||
|
f. "Materials" shall mean, collectively, Alibaba Cloud's proprietary Qwen and Documentation (and any portion thereof) made available under this Agreement.
|
||||||
|
g. "Source" form shall mean the preferred form for making modifications, including but not limited to model source code, documentation source, and configuration files.
|
||||||
|
h. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
|
||||||
|
i. "Non-Commercial" shall mean for research or evaluation purposes only.
|
||||||
|
|
||||||
|
2. Grant of Rights
|
||||||
|
a. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under Alibaba Cloud's intellectual property or other rights owned by us embodied in the Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the Materials FOR NON-COMMERCIAL PURPOSES ONLY.
|
||||||
|
b. If you are commercially using the Materials, you shall request a license from us.
|
||||||
|
|
||||||
|
3. Redistribution
|
||||||
|
You may distribute copies or make the Materials, or derivative works thereof, available as part of a product or service that contains any of them, with or without modifications, and in Source or Object form, provided that you meet the following conditions:
|
||||||
|
a. You shall give any other recipients of the Materials or derivative works a copy of this Agreement;
|
||||||
|
b. You shall cause any modified files to carry prominent notices stating that you changed the files;
|
||||||
|
c. You shall retain in all copies of the Materials that you distribute the following attribution notices within a "Notice" text file distributed as a part of such copies: "Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) Alibaba Cloud. All Rights Reserved."; and
|
||||||
|
d. You may add your own copyright statement to your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of your modifications, or for any such derivative works as a whole, provided your use, reproduction, and distribution of the work otherwise complies with the terms and conditions of this Agreement.
|
||||||
|
|
||||||
|
4. Rules of use
|
||||||
|
a. The Materials may be subject to export controls or restrictions in China, the United States or other countries or regions. You shall comply with applicable laws and regulations in your use of the Materials.
|
||||||
|
b. If you use the Materials or any outputs or results therefrom to create, train, fine-tune, or improve an AI model that is distributed or made available, you shall prominently display “Built with Qwen” or “Improved using Qwen” in the related product documentation.
|
||||||
|
|
||||||
|
5. Intellectual Property
|
||||||
|
a. We retain ownership of all intellectual property rights in and to the Materials and derivatives made by or for us. Conditioned upon compliance with the terms and conditions of this Agreement, with respect to any derivative works and modifications of the Materials that are made by you, you are and will be the owner of such derivative works and modifications.
|
||||||
|
b. No trademark license is granted to use the trade names, trademarks, service marks, or product names of us, except as required to fulfill notice requirements under this Agreement or as required for reasonable and customary use in describing and redistributing the Materials.
|
||||||
|
c. If you commence a lawsuit or other proceedings (including a cross-claim or counterclaim in a lawsuit) against us or any entity alleging that the Materials or any output therefrom, or any part of the foregoing, infringe any intellectual property or other right owned or licensable by you, then all licenses granted to you under this Agreement shall terminate as of the date such lawsuit or other proceeding is commenced or brought.
|
||||||
|
6. Disclaimer of Warranty and Limitation of Liability
|
||||||
|
a. We are not obligated to support, update, provide training for, or develop any further version of the Qwen Materials or to grant any license thereto.
|
||||||
|
b. THE MATERIALS ARE PROVIDED "AS IS" WITHOUT ANY EXPRESS OR IMPLIED WARRANTY OF ANY KIND INCLUDING WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT, OR FITNESS FOR A PARTICULAR PURPOSE. WE MAKE NO WARRANTY AND ASSUME NO RESPONSIBILITY FOR THE SAFETY OR STABILITY OF THE MATERIALS AND ANY OUTPUT THEREFROM.
|
||||||
|
c. IN NO EVENT SHALL WE BE LIABLE TO YOU FOR ANY DAMAGES, INCLUDING, BUT NOT LIMITED TO ANY DIRECT, OR INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING FROM YOUR USE OR INABILITY TO USE THE MATERIALS OR ANY OUTPUT OF IT, NO MATTER HOW IT’S CAUSED.
|
||||||
|
d. You will defend, indemnify and hold harmless us from and against any claim by any third party arising out of or related to your use or distribution of the Materials.
|
||||||
|
|
||||||
|
7. Survival and Termination.
|
||||||
|
a. The term of this Agreement shall commence upon your acceptance of this Agreement or access to the Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein.
|
||||||
|
b. We may terminate this Agreement if you breach any of the terms or conditions of this Agreement. Upon termination of this Agreement, you must delete and cease use of the Materials. Sections 6 and 8 shall survive the termination of this Agreement.
|
||||||
|
|
||||||
|
8. Governing Law and Jurisdiction.
|
||||||
|
a. This Agreement and any dispute arising out of or relating to it will be governed by the laws of China, without regard to conflict of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement.
|
||||||
|
b. The People's Courts in Hangzhou City shall have exclusive jurisdiction over any dispute arising out of this Agreement.
|
||||||
|
9. Other Terms and Conditions.
|
||||||
|
a. Any arrangements, understandings, or agreements regarding the Material not stated herein are separate from and independent of the terms and conditions of this Agreement. You shall request a separate license from us, if you use the Materials in ways not expressly agreed to in this Agreement.
|
||||||
|
b. We shall not be bound by any additional or different terms or conditions communicated by you unless expressly agreed.
|
||||||
668
README.md
Normal file
668
README.md
Normal file
@@ -0,0 +1,668 @@
|
|||||||
|
---
|
||||||
|
license_name: qwen-research
|
||||||
|
license_link: https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE
|
||||||
|
datasets:
|
||||||
|
- remyxai/SpaceThinker
|
||||||
|
base_model:
|
||||||
|
- UCSC-VLAA/VLAA-Thinker-Qwen2.5VL-3B
|
||||||
|
tags:
|
||||||
|
- remyx
|
||||||
|
- qwen2.5-vl
|
||||||
|
- spatial-reasoning
|
||||||
|
- multimodal
|
||||||
|
- vlm
|
||||||
|
- vqasynth
|
||||||
|
- thinking
|
||||||
|
- reasoning
|
||||||
|
- test-time-compute
|
||||||
|
- robotics
|
||||||
|
- embodied-ai
|
||||||
|
- quantitative-spatial-reasoning
|
||||||
|
- distance-estimation
|
||||||
|
- visual-question-answering
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
pipeline_tag: image-text-to-text
|
||||||
|
library_name: transformers
|
||||||
|
model-index:
|
||||||
|
- name: SpaceThinker-Qwen2.5VL-3B
|
||||||
|
results:
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: Q-Spatial-Bench
|
||||||
|
type: custom
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.3226
|
||||||
|
results_by_distance_bucket:
|
||||||
|
- name: 0-10cm
|
||||||
|
count: 7
|
||||||
|
successes: 3
|
||||||
|
success_rate: 0.4286
|
||||||
|
- name: 10-30cm
|
||||||
|
count: 28
|
||||||
|
successes: 5
|
||||||
|
success_rate: 0.1786
|
||||||
|
- name: 30-60cm
|
||||||
|
count: 16
|
||||||
|
successes: 8
|
||||||
|
success_rate: 0.5
|
||||||
|
- name: 60-100cm
|
||||||
|
count: 17
|
||||||
|
successes: 9
|
||||||
|
success_rate: 0.5294
|
||||||
|
- name: 100-200cm
|
||||||
|
count: 19
|
||||||
|
successes: 4
|
||||||
|
success_rate: 0.2105
|
||||||
|
- name: 200cm+
|
||||||
|
count: 6
|
||||||
|
successes: 1
|
||||||
|
success_rate: 0.1667
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: 3DSRBench
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.2052
|
||||||
|
results_by_category:
|
||||||
|
- name: 3D Positional Relation / Orientation
|
||||||
|
success_rate: 0.2603
|
||||||
|
- name: Object Localization / 3D Localization
|
||||||
|
success_rate: 0.3493
|
||||||
|
- name: Object Properties / Size
|
||||||
|
success_rate: 0.0058
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: BLINK
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.2841
|
||||||
|
results_by_category:
|
||||||
|
- name: 3D Positional Relation / Orientation
|
||||||
|
success_rate: 0.5874
|
||||||
|
- name: Counting / Object Counting
|
||||||
|
success_rate: 0.45
|
||||||
|
- name: Depth and Distance / Relative
|
||||||
|
success_rate: 0
|
||||||
|
- name: Object Localization / 2D Localization
|
||||||
|
success_rate: 0.041
|
||||||
|
- name: Point and Object Tracking / Point Correspondence
|
||||||
|
success_rate: 0.0581
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: MMIU
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.2651
|
||||||
|
results_by_category:
|
||||||
|
- name: Camera and Image Transformation / 2D Transformation
|
||||||
|
success_rate: 0.1325
|
||||||
|
- name: Camera and Image Transformation / 3D Camera Pose
|
||||||
|
success_rate: 0.28
|
||||||
|
- name: Camera and Image Transformation / Camera Motion
|
||||||
|
success_rate: 0.4436
|
||||||
|
- name: Depth and Distance / Absolute
|
||||||
|
success_rate: 0.005
|
||||||
|
- name: Object Localization / 3D Localization
|
||||||
|
success_rate: 0.2275
|
||||||
|
- name: Point and Object Tracking / 3D Tracking
|
||||||
|
success_rate: 0.505
|
||||||
|
- name: Point and Object Tracking / Point Correspondence
|
||||||
|
success_rate: 0.265
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: MMVP
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.5367
|
||||||
|
results_by_category:
|
||||||
|
- name: Others / Miscellaneous
|
||||||
|
success_rate: 0.5367
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: QSpatialBench-Plus
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.5149
|
||||||
|
results_by_category:
|
||||||
|
- name: Depth and Distance / Absolute
|
||||||
|
success_rate: 0.5149
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: QSpatialBench-ScanNet
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.5733
|
||||||
|
results_by_category:
|
||||||
|
- name: Depth and Distance / Absolute
|
||||||
|
success_rate: 0.48
|
||||||
|
- name: Object Properties / Size
|
||||||
|
success_rate: 0.6667
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: RealWorldQA
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.5699
|
||||||
|
results_by_category:
|
||||||
|
- name: Others / Miscellaneous
|
||||||
|
success_rate: 0.5699
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: SpatialSense
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.6405
|
||||||
|
results_by_category:
|
||||||
|
- name: 3D Positional Relation / Orientation
|
||||||
|
success_rate: 0.6405
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: VGBench
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.3257
|
||||||
|
results_by_category:
|
||||||
|
- name: Camera and Image Transformation / 2D Transformation
|
||||||
|
success_rate: 0.2258
|
||||||
|
- name: Camera and Image Transformation / 3D Camera Pose
|
||||||
|
success_rate: 0.401
|
||||||
|
- name: Depth and Distance / Absolute
|
||||||
|
success_rate: 0.3904
|
||||||
|
- name: Depth and Distance / Relative
|
||||||
|
success_rate: 0.2127
|
||||||
|
- name: Object Localization / 3D Localization
|
||||||
|
success_rate: 0.4217
|
||||||
|
- name: Point and Object Tracking / 3D Tracking
|
||||||
|
success_rate: 0.3027
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: VSI-Bench_8
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.2781
|
||||||
|
results_by_category:
|
||||||
|
- name: 3D Positional Relation / Orientation
|
||||||
|
success_rate: 0.3864
|
||||||
|
- name: Counting / Object Counting
|
||||||
|
success_rate: 0.2246
|
||||||
|
- name: Depth and Distance / Absolute
|
||||||
|
success_rate: 0.0639
|
||||||
|
- name: Depth and Distance / Relative
|
||||||
|
success_rate: 0.3479
|
||||||
|
- name: Object Properties / Size
|
||||||
|
success_rate: 0.143
|
||||||
|
- name: Others / Miscellaneous
|
||||||
|
success_rate: 0.3042
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: VSR-ZeroShot
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.6997
|
||||||
|
results_by_category:
|
||||||
|
- name: 3D Positional Relation / Orientation
|
||||||
|
success_rate: 0.6997
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: cvbench
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.2457
|
||||||
|
results_by_category:
|
||||||
|
- name: Counting / Object Counting
|
||||||
|
success_rate: 0.4886
|
||||||
|
- name: Depth and Distance / Relative
|
||||||
|
success_rate: 0.2025
|
||||||
|
- name: Object Localization / 3D Localization
|
||||||
|
success_rate: 0.0462
|
||||||
|
- task:
|
||||||
|
type: visual-question-answering
|
||||||
|
name: Spatial Reasoning
|
||||||
|
dataset:
|
||||||
|
name: spatialbench
|
||||||
|
type: benchmark
|
||||||
|
metrics:
|
||||||
|
- type: success_rate
|
||||||
|
name: Overall Success Rate
|
||||||
|
value: 0.565
|
||||||
|
results_by_category:
|
||||||
|
- name: 3D Positional Relation / Orientation
|
||||||
|
success_rate: 0.5
|
||||||
|
- name: Counting / Object Counting
|
||||||
|
success_rate: 0.7
|
||||||
|
- name: Object Properties / Existence
|
||||||
|
success_rate: 0.825
|
||||||
|
- name: Object Properties / Reachability
|
||||||
|
success_rate: 0.575
|
||||||
|
- name: Object Properties / Size
|
||||||
|
success_rate: 0.225
|
||||||
|
new_version: remyxai/SpaceOm
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
[](https://remyx.ai/?model_id=SpaceThinker-Qwen2.5VL-3B&sha256=abc123def4567890abc123def4567890abc123def4567890abc123def4567890)
|
||||||
|
|
||||||
|
# SpaceThinker-Qwen2.5VL-3B
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## 📚 Contents
|
||||||
|
|
||||||
|
- [🚀 Try It Live](#try-the-spacethinker-space)
|
||||||
|
- [🧠 Model Overview](#model-overview)
|
||||||
|
- [📏 Quantitative Spatial Reasoning](#spatial-reasoning-capabilities)
|
||||||
|
- [🔍 View Examples](#examples-of-spacethinker)
|
||||||
|
- [📊 Evaluation & Benchmarks](#model-evaluation)
|
||||||
|
- [🏃♀️ Running SpaceThinker](#running-spacethinker)
|
||||||
|
- [🏋️♂️ Training Configuration](#training-spacethinker)
|
||||||
|
- [📂 Dataset Info](#spacethinker-dataset)
|
||||||
|
- [⚠️ Limitations](#limitations)
|
||||||
|
- [📜 Citation](#citation)
|
||||||
|
|
||||||
|
|
||||||
|
## Try the SpaceThinker Space
|
||||||
|
|
||||||
|
[](https://huggingface.co/spaces/remyxai/SpaceThinker-Qwen2.5VL-3B)
|
||||||
|
|
||||||
|
|
||||||
|
# Model Overview
|
||||||
|
|
||||||
|
**SpaceThinker-Qwen2.5VL-3B** is a thinking/reasoning multimodal/vision-language model (VLM) trained to enhance spatial reasoning with test-time compute by fine-tuning
|
||||||
|
`UCSC-VLAA/VLAA-Thinker-Qwen2.5VL-3B` on synthetic reasoning traces generated by the [VQASynth](https://huggingface.co/datasets/remyxai/SpaceThinker) pipeline.
|
||||||
|
|
||||||
|
|
||||||
|
- **Model Type:** Multimodal, Vision-Language Model
|
||||||
|
- **Architecture**: `Qwen2.5-VL-3B`
|
||||||
|
- **Model Size:** 3.75B parameters (FP16)
|
||||||
|
- **Finetuned from:** `UCSC-VLAA/VLAA-Thinker-Qwen2.5VL-3B`
|
||||||
|
- **Finetune Strategy:** LoRA (Low-Rank Adaptation)
|
||||||
|
- **License:** Apache-2.0
|
||||||
|
|
||||||
|
Check out the [SpaceThinker collection](https://huggingface.co/collections/remyxai/spacethinker-68014f174cd049ca5acca4e5)
|
||||||
|
|
||||||
|
## Spatial Reasoning Capabilities
|
||||||
|
|
||||||
|
Strong quantitative spatial reasoning is critical for embodied AI applications demanding the ability to plan and navigate a 3D space, such as robotics and drones.
|
||||||
|
**SpaceThinker** improves capabilities using test-time compute, trained with samples which ground the final response on a consistent explanation of a collection of scene observations.
|
||||||
|
|
||||||
|
- Enhanced Quantitative Spatial Reasoning (e.g., distances, sizes)
|
||||||
|
- Grounded object relations (e.g., left-of, above, inside)
|
||||||
|
|
||||||
|
### Examples of SpaceThinker
|
||||||
|
|
||||||
|
Consider the following sample images/prompts for visual QA of **SpaceThinker-Qwen2.5VL-3B**
|
||||||
|
|
||||||
|
| <img src="images/example_1.jpg" alt="sample_1" style="max-height: 350px;" > | <img src="images/example_2.png" alt="sample_2" style="max-height: 350px;"> | <img src="images/example_3.png" alt="sample_3" style="max-height: 350px;"> |
|
||||||
|
|----------|-------------|--------|
|
||||||
|
| **Prompt:** How far is the man in the red hat from the pallet of boxes in feet? | **Prompt:** How far is the Goal from the penalty taker in this image? | **Prompt:** Approximately how many meters apart are the chair and bookshelf? |
|
||||||
|
| **Reasoning:** To determine how far the man in the red hat is from the pallet of boxes, I need to consider the spatial arrangement in the warehouse. The man is walking on the floor, and there's a visible gap between him and the pallet. First, I'll estimate the size of the pallet. A standard pallet is typically around 4 feet wide and 8 feet long. Given that the man is standing near the edge of the pallet, he might be about 2–3 feet away from the front edge of the pallet. Next, I'll consider the distance from the pallet to the man. The man appears to be walking along the aisle, which seems to be about 6–7 feet wide. If he is standing at the edge of the pallet, he would likely be about 2–3 feet from the front edge of the pallet. Combining these observations, the man in the red hat is approximately 2–3 feet from the pallet of boxes. | **Reasoning:** First, I observe the scene where a player in a red uniform stands near the goal. The goal itself is a standard soccer goalpost, which typically extends 2.5 meters beyond the line of the goalposts. The player appears to be standing close to the goal, but not directly at the goal line. To estimate the distance, I consider that the player's position relative to the goal suggests he is likely standing closer than 2.5 meters away. Given the typical setup of a penalty kick scenario, the player would be positioned further back from the goal than the goal itself. By visual estimation, the player seems to be standing approximately 10–15 meters from the goal. This range aligns with the common distance for a penalty kick, considering the player's stance and the goal's structure. Thus, the distance from the goal to the penalty taker is likely around 10–15 meters. | **Reasoning:** To determine the distance between the chair and the bookshelf, I'll consider the typical dimensions of these objects. A standard office chair is around 60–70 cm tall, and a bookshelf can be anywhere from 1.2 to 1.8 meters tall. Given that the chair is likely placed on a desk or floor level, the height difference between the chair and the top of the bookshelf would be approximately 1 meter. Assuming the bookshelf is at least 1 meter high, and the chair is at ground level, the vertical distance between them is indeed 1 meter. The horizontal distance might vary depending on the layout, but given the room's dimensions and the placement of the chair relative to the bookshelf, the total distance between the chair and the top of the bookshelf is likely around 1 meter. Therefore, the chair and bookshelf are approximately 1 meter apart. |
|
||||||
|
| **Answer:** 2–3 feet | **Answer:** 10–15 meters | **Answer:** 1 meter |
|
||||||
|
|
||||||
|
|
||||||
|
> **SpaceThinker** demonstrates grounded, quantitative spatial reasoning—inferring accurate distances, interpreting 3D scene context, and formatting open-ended answers precisely
|
||||||
|
> by integrating visual cues, real-world object priors, and human-centric spatial logic.
|
||||||
|
|
||||||
|
Read more about using test-time compute [here](https://huggingface.co/spaces/open-r1/README/discussions/10) for enhanced multimodal quantitative spatial reasoning.
|
||||||
|
|
||||||
|
## Running SpaceThinker
|
||||||
|
|
||||||
|
### Space
|
||||||
|
Try the **SpaceThinker** Space
|
||||||
|
|
||||||
|
[](https://huggingface.co/spaces/remyxai/SpaceThinker-Qwen2.5VL-3B)
|
||||||
|
|
||||||
|
### Ollama
|
||||||
|
To launch with ollama, run:
|
||||||
|
```bash
|
||||||
|
ollama run hf.co/remyxai/SpaceThinker-Qwen2.5VL-3B:latest
|
||||||
|
```
|
||||||
|
or
|
||||||
|
```bash
|
||||||
|
ollama run remyxai/spacethinker
|
||||||
|
```
|
||||||
|
|
||||||
|
### llama.cpp
|
||||||
|
To run locally with **llama.cpp**, install and build this [branch](https://github.com/HimariO/llama.cpp.qwen2.5vl/tree/qwen25-vl) and download the [.gguf weights here](https://huggingface.co/remyxai/SpaceThinker-Qwen2.5VL-3B/tree/main/gguf)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./llama-qwen2vl-cli -m spacethinker-qwen2.5VL-3B-F16.gguf
|
||||||
|
--mmproj spacethinker-qwen2.5vl-3b-vision.gguf
|
||||||
|
--image images/example_1.jpg --threads 24 -ngl 9
|
||||||
|
-p "Does the man in blue shirt working have a greater \\
|
||||||
|
height compared to the wooden pallet with boxes on floor?"
|
||||||
|
```
|
||||||
|
|
||||||
|
Run using **llama.cpp in colab**
|
||||||
|
|
||||||
|
[](https://colab.research.google.com/drive/1_ShhJAqnac8L4N9o1YNdsxCksSLJCrU7?usp=sharing)
|
||||||
|
|
||||||
|
### Transformers
|
||||||
|
Run locally using **Transformers**
|
||||||
|
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
from PIL import Image
|
||||||
|
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
|
||||||
|
import requests
|
||||||
|
from io import BytesIO
|
||||||
|
|
||||||
|
# Configuration
|
||||||
|
model_id = "remyxai/SpaceThinker-Qwen2.5VL-3B"
|
||||||
|
image_path = "images/example_1.jpg" # or local path
|
||||||
|
prompt = "What can you infer from this image about the environment?"
|
||||||
|
system_message = (
|
||||||
|
"You are VL-Thinking 🤔, a helpful assistant with excellent reasoning ability. "
|
||||||
|
"You should first think about the reasoning process and then provide the answer. "
|
||||||
|
"Use <think>...</think> and <answer>...</answer> tags."
|
||||||
|
)
|
||||||
|
|
||||||
|
# Load model and processor
|
||||||
|
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
|
||||||
|
model_id, device_map="auto", torch_dtype=torch.bfloat16
|
||||||
|
)
|
||||||
|
processor = AutoProcessor.from_pretrained(model_id)
|
||||||
|
|
||||||
|
# Load and preprocess image
|
||||||
|
if image_path.startswith("http"):
|
||||||
|
image = Image.open(BytesIO(requests.get(image_path).content)).convert("RGB")
|
||||||
|
else:
|
||||||
|
image = Image.open(image_path).convert("RGB")
|
||||||
|
if image.width > 512:
|
||||||
|
ratio = image.height / image.width
|
||||||
|
image = image.resize((512, int(512 * ratio)), Image.Resampling.LANCZOS)
|
||||||
|
|
||||||
|
# Format input
|
||||||
|
chat = [
|
||||||
|
{"role": "system", "content": [{"type": "text", "text": system_message}]},
|
||||||
|
{"role": "user", "content": [{"type": "image", "image": image},
|
||||||
|
{"type": "text", "text": prompt}]}
|
||||||
|
]
|
||||||
|
text_input = processor.apply_chat_template(chat, tokenize=False,
|
||||||
|
add_generation_prompt=True)
|
||||||
|
|
||||||
|
# Tokenize
|
||||||
|
inputs = processor(text=[text_input], images=[image],
|
||||||
|
return_tensors="pt").to("cuda")
|
||||||
|
|
||||||
|
# Generate response
|
||||||
|
generated_ids = model.generate(**inputs, max_new_tokens=1024)
|
||||||
|
output = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
||||||
|
|
||||||
|
print("Response:\n", output)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## SpaceThinker Dataset
|
||||||
|
|
||||||
|
The **SpaceThinker** dataset includes over 12K samples synthesized using VQASynth on a subset of images in the localized narratives split of [the cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron).
|
||||||
|
**SpaceThinker** is formatted similar to the [Llama-Nemotron-Post-Training-Dataset-v1](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset) to toggle reasoning.
|
||||||
|
|
||||||
|
|
||||||
|
The model builds upon the ideas from [SpatialVLM (Chen et al., 2024)](https://spatial-vlm.github.io/), introducing synthetic reasoning traces grounded on a 3D scene reconstruction pipeline using **Molmo, VGGT, SAM2**.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
**Dataset Summary**
|
||||||
|
- ~12K synthetic spatial reasoning traces
|
||||||
|
- Question types: spatial relations (distances (units), above, left-of, contains, closest to)
|
||||||
|
- Format: image (RGB) + question + answer with reasoning traces
|
||||||
|
- Dataset: [remyxai/SpaceThinker](https://huggingface.co/datasets/remyxai/SpaceThinker)
|
||||||
|
- Code: [Synthetize Spatial Reasoning Traces with VQASynth](https://github.com/remyxai/VQASynth)
|
||||||
|
|
||||||
|
|
||||||
|
## Training SpaceThinker
|
||||||
|
|
||||||
|
**PEFT Configuration**
|
||||||
|
- Architecture: Qwen2.5-VL-3B
|
||||||
|
- Base model: UCSC-VLAA/VLAA-Thinker-Qwen2.5VL-3B
|
||||||
|
- Method: LoRA finetuning (PEFT)
|
||||||
|
- LoRA Alpha: 256
|
||||||
|
- LoRA Rank: 128
|
||||||
|
- Target Modules: q_proj, v_proj
|
||||||
|
- Optimizer: AdamW (lr=2e-5), batch size = 1, epochs = 3
|
||||||
|
- Max input length: 1024 tokens
|
||||||
|
|
||||||
|
Reproduce LoRA SFT training with included script:
|
||||||
|
```bash
|
||||||
|
python train.py
|
||||||
|
```
|
||||||
|
Wandb logs available [here](https://wandb.ai/smellslikeml/qwen2.5-3b-instruct-trl-sft-spacethinker).
|
||||||
|
|
||||||
|
|
||||||
|
## Model Evaluation
|
||||||
|
|
||||||
|
### OmniSpatial
|
||||||
|
|
||||||
|
See how SpaceThinker compares on **[OmniSpatial](https://www.arxiv.org/pdf/2506.03135)** - dynamic reasoning, complex spatial logic, spatial interaction, and perspective-taking.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### SpatialScore
|
||||||
|
|
||||||
|
Evaluate Spatial Reasoning with the **SpatialScore** Benchmark.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Evaluate **SpaceThinker** on the [SpatialScore](https://haoningwu3639.github.io/SpatialScore/) benchmarks for general spatial reasoning in the following colab notebook:
|
||||||
|
|
||||||
|
[](https://colab.research.google.com/drive/1eRc5_vpUCS4QxwzBNAvKi0z0IANdC9N2?usp=sharing)
|
||||||
|
|
||||||
|
The following chart compares performance between **SpaceThinker** and **SpaceQwen** on the **SpatialScore** benchmarks sources.
|
||||||
|
|
||||||
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/647777304ae93470ffc28913/HYR3temzQ8YjdKQWTtjxt.png" alt="SpaceThinker_v_SpaceQwen" style="max-height: 250px;">
|
||||||
|
|
||||||
|
Find the full summary of the [results here](https://huggingface.co/datasets/salma-remyx/SpaceThinker_SpatialScore).
|
||||||
|
|
||||||
|
**SpaceThinker** shines on the **SpatialScore-Hard** split:
|
||||||
|
|
||||||
|
| **Model** | **Overall** | **Count.** | **Obj-Loc.** | **Pos-Rel.** | **Dist.** | **Obj-Prop.** | **Cam.\&IT.** | **Tracking** | **Others** |
|
||||||
|
| ------------------- | ----------- | ---------- | ------------ | ------------ | --------- | ------------- | ------------- | ------------ | ---------- |
|
||||||
|
| SpaceOm | 18.83 | 17.11 | 12.57 | 21.50 | 23.66 | 31.94 | 18.86 | 14.79 | 9.14 |
|
||||||
|
| 🧠 **SpaceThinker** | 19.47 | 17.82 | 12.57 | 21.50 | 25.37 | 30.23 | 22.86 | 14.79 | 9.71 |
|
||||||
|
| SpaceQwen2.5VL-3B | 17.21 | 20.42 | 18.86 | 27.10 | 6.86 | 12.57 | 8.57 | 23.08 | 18.86 |
|
||||||
|
| SpatialBot-Phi2-3B | 19.87 | 19.72 | 27.59 | 25.23 | 8.00 | 23.43 | 13.71 | 17.16 | 22.86 |
|
||||||
|
| Kimi-VL-3B | 16.29 | 12.68 | 18.86 | 12.62 | 17.71 | 22.29 | 18.86 | 11.83 | 15.43 |
|
||||||
|
| Kimi-VL-3B-Thinking | 26.36 | 19.72 | 28.00 | 38.79 | 27.43 | 32.57 | 14.86 | 18.34 | 26.86 |
|
||||||
|
| Qwen2.5-VL-3B | 16.86 | 18.31 | 7.43 | 16.82 | 19.43 | 31.43 | 15.43 | 14.79 | 11.43 |
|
||||||
|
| InternVL2.5-4B | 13.71 | 6.34 | 21.71 | 15.42 | 14.86 | 12.00 | 16.00 | 12.43 | 9.14 |
|
||||||
|
|
||||||
|
|
||||||
|
See [all results](https://huggingface.co/datasets/salma-remyx/SpaceThinker_SpatialScore-Hard/blob/main/all_results.json) for **SpatialScore-Hard**.
|
||||||
|
|
||||||
|
|
||||||
|
### SpaCE-10
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
| **Model** | **Overall** | **EQ** | **SQ** | **SA** | **OO** | **OS** | **EP** | **FR** | **SP** | **Source** |
|
||||||
|
|--------------------------|-------------|----------|----------|----------|----------|----------|----------|----------|----------|-------------|
|
||||||
|
| InternVL2.5-4B | **36.01** | **34.30**| 34.40 | 43.60 | 44.40 | 16.50 | **31.10**| **50.10**| **33.70**| Table |
|
||||||
|
| 🧠**SpaceThinker** | 32.72 | 32.73 | 24.81 | 47.26 | 50.33 | 33.63 | 9.25 | 37.54 | 26.25 | GPT Eval |
|
||||||
|
| SpaceOm | 32.32 | 32.47 | 24.81 | **47.63**| 50.00 | 32.52 | 9.12 | 37.04 | 25.00 | GPT Eval |
|
||||||
|
| SpaceQwen | 31.98 | 31.19 | 25.89 | 41.61 | **51.98**| **35.18**| 10.97 | 36.54 | 22.50 | GPT Eval |
|
||||||
|
| Qwen2.5-VL-3B-Instruct | 30.00 | 31.70 | **45.50**| 39.00 | 43.00 | 25.30 | 11.50 | 22.80 | 21.20 | Table |
|
||||||
|
|
||||||
|
**Legend:**
|
||||||
|
- EQ: Entity Quantification
|
||||||
|
- SQ: Scene Quantification
|
||||||
|
- SA: Size Assessment
|
||||||
|
- OO: Object-Object spatial relations
|
||||||
|
- OS: Object-Scene spatial relations
|
||||||
|
- EP: Entity Presence
|
||||||
|
- FR: Functional Reasoning
|
||||||
|
- SP: Spatial Planning
|
||||||
|
|
||||||
|
> ℹ️ Note: Scores for SpaceQwen, SpaceThinker, SpaceOm are generated via `gpt_eval_score` on single-choice (`*-single`) versions of the SpaCE-10 benchmark tasks. Other entries reflect leaderboard accuracy scores from the official SpaCE-10 evaluation table.
|
||||||
|
|
||||||
|
Read more about the [SpaCE-10 benchmark](https://arxiv.org/pdf/2506.07966v1) or see [results here](https://huggingface.co/datasets/salma-remyx/SpaceThinker_SpaCE-10_Results/blob/main/20250612_062336_results.json)
|
||||||
|
|
||||||
|
## QSpatial++ Comparison Table (4/25/25)
|
||||||
|
|
||||||
|
| **Model** | **SpaceThinker-Qwen2.5VL-3B** | **gpt-4o** | **gemini-2.5-pro-preview-03-25** |
|
||||||
|
|------------------------|----------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------|
|
||||||
|
| **QSpatial++ Predictions** | <img src="https://cdn-uploads.huggingface.co/production/uploads/647777304ae93470ffc28913/W4b6fV0AxT6GsYR1XiQvA.png" alt="SpaceThinker sample" style="max-height: 150px;"> | <img src="https://cdn-uploads.huggingface.co/production/uploads/647777304ae93470ffc28913/j_NFQ9Lns8ON9Rzg3Fs0r.png" alt="gpt-4o sample" style="max-height: 150px;"> | <img src="https://cdn-uploads.huggingface.co/production/uploads/647777304ae93470ffc28913/Ot64jVvwdldpCuDr-6PjX.png" alt="Gemini sample" style="max-height: 150px;"> |
|
||||||
|
| **Colab Notebook** | [](https://colab.research.google.com/drive/1buEe2QC4_pnrJwQ9XyRAH7RfaIa6pbex?usp=sharing) | [](https://colab.research.google.com/drive/1zNv41ONUeoaEigscz9muZ3eVFtxev0qv?usp=sharing) | [](https://colab.research.google.com/drive/153bbTxrRBH52n74jONvpsbqJ1XYNByKw?usp=sharing) |
|
||||||
|
| **Success Rate (%) ↑** | **55** | 43 | 52 |
|
||||||
|
| **Samples Completed ↑**| **99 / 100** | 95 / 100 | **99 / 100** |
|
||||||
|
| **sMAPE (%) ↓** | 66 | 71 | **62** |
|
||||||
|
|
||||||
|
### Metric Notes
|
||||||
|
- **Success Rate (%)**: Higher is better ↑
|
||||||
|
- **Samples Completed**: Higher is better ↑
|
||||||
|
- **sMAPE (%)**: Lower is better ↓
|
||||||
|
|
||||||
|
Try distance estimation focusing on **Q-Spatial-Bench** in the colab notebook here:
|
||||||
|
|
||||||
|
[](https://colab.research.google.com/drive/1NH2n-PRJJOiu_md8agyYCnxEZDGO5ICJ?usp=sharing)
|
||||||
|
|
||||||
|
The [Q-Spatial-Bench dataset](https://huggingface.co/datasets/andrewliao11/Q-Spatial-Bench) includes hundreds of
|
||||||
|
VQA samples designed to evaluate quantitative spatial reasoning of VLMs with high-precision.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Using the Colab notebook we evaluate **SpaceThinker** on the **QSpatial++** split under two conditions:
|
||||||
|
|
||||||
|
|
||||||
|
- **Default System Prompt**:
|
||||||
|
- Prompts completed: **93 / 101**
|
||||||
|
- Correct answers: **30**
|
||||||
|
- **Accuracy**: **32.26%**
|
||||||
|
|
||||||
|
- **Prompting for step-by-step reasoning** using the [spatial prompt](https://github.com/andrewliao11/Q-Spatial-Bench-code/blob/main/prompt_templates/spatial_prompt_steps.txt) from **Q-Spatial-Bench**:
|
||||||
|
- Correct answers: **53**
|
||||||
|
- **Accuracy**: **52.48%**
|
||||||
|
|
||||||
|
Using the spatial prompt improves the number of correct answers and overall accuracy rate while improving the task completion rate.
|
||||||
|
|
||||||
|
Updating the comparison from **Q-Spatial-Bench** [project page](https://andrewliao11.github.io/spatial_prompt/), the **SpaceThinker-Qwen2.5-VL-3B** VLM using
|
||||||
|
the SpatialPrompt for step-by-step reasoning performs on par with larger, closed, frontier API providers.
|
||||||
|
|
||||||
|
|
||||||
|
The following chart makes further comparisons to assess prompt sensitivity by evaluating w/o the benefit of the
|
||||||
|
optimized step-by-step instructions. This comparison helps to quantify the effect of reasoning versus non-reasoning
|
||||||
|
models as well as that of SFT by LoRA with synthetic spatial reasoning data.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Consider the extended [comparisons here](https://huggingface.co/datasets/salma-remyx/Q-Spatial-Bench-sMAPE-Comparison) sweeping additional model sizes and architectures.
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- Performance may degrade in cluttered environments or camera perspective.
|
||||||
|
- This model was fine-tuned using synthetic reasoning over an internet image dataset.
|
||||||
|
- Multimodal biases inherent to the base model (Qwen2.5-VL) may persist.
|
||||||
|
- Not intended for use in safety-critical or legal decision-making.
|
||||||
|
|
||||||
|
> Users are encouraged to evaluate outputs critically and consider fine-tuning for domain-specific safety and performance. Distances estimated using autoregressive
|
||||||
|
> transformers may help in higher-order reasoning for planning and behavior but may not be suitable replacements for measurements taken with high-precision sensors,
|
||||||
|
> calibrated stereo vision systems, or specialist monocular depth estimation models capable of more accurate, pixel-wise predictions and real-time performance.
|
||||||
|
|
||||||
|
|
||||||
|
## Citation
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
@article{chen2024spatialvlm,
|
||||||
|
title = {SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities},
|
||||||
|
author = {Chen, Boyuan and Xu, Zhuo and Kirmani, Sean and Ichter, Brian and Driess, Danny and Florence, Pete and Sadigh, Dorsa and Guibas, Leonidas and Xia, Fei},
|
||||||
|
journal = {arXiv preprint arXiv:2401.12168},
|
||||||
|
year = {2024},
|
||||||
|
url = {https://arxiv.org/abs/2401.12168},
|
||||||
|
}
|
||||||
|
|
||||||
|
@misc{qwen2.5-VL,
|
||||||
|
title = {Qwen2.5-VL},
|
||||||
|
url = {https://qwenlm.github.io/blog/qwen2.5-vl/},
|
||||||
|
author = {Qwen Team},
|
||||||
|
month = {January},
|
||||||
|
year = {2025}
|
||||||
|
}
|
||||||
|
|
||||||
|
@misc{vl-thinking2025,
|
||||||
|
title={SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models },
|
||||||
|
author={Hardy Chen and Haoqin Tu and Fali Wang and Hui Liu and Xianfeng Tang and Xinya Du and Yuyin Zhou and Cihang Xie},
|
||||||
|
year = {2025},
|
||||||
|
publisher = {GitHub},
|
||||||
|
journal = {GitHub repository},
|
||||||
|
howpublished = {\url{https://github.com/UCSC-VLAA/VLAA-Thinking}},
|
||||||
|
}
|
||||||
|
|
||||||
|
@inproceedings{
|
||||||
|
liaos2024reasoning,
|
||||||
|
title={Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models},
|
||||||
|
author={Yuan-Hong Liao and Rafid Mahmood and Sanja Fidler and David Acuna},
|
||||||
|
booktitle={The 2024 Conference on Empirical Methods in Natural Language Processing},
|
||||||
|
year={2024},
|
||||||
|
url={https://arxiv.org/abs/2409.09788},
|
||||||
|
}
|
||||||
|
|
||||||
|
@article{wu2025spatialscore,
|
||||||
|
author = {Wu, Haoning and Huang, Xiao and Chen, Yaohui and Zhang, Ya and Wang, Yanfeng and Xie, Weidi},
|
||||||
|
title = {SpatialScore: Towards Unified Evaluation for Multimodal Spatial Understanding},
|
||||||
|
journal = {arXiv preprint arXiv:2505.17012},
|
||||||
|
year = {2025},
|
||||||
|
}
|
||||||
|
|
||||||
|
@article{omnispatial25,
|
||||||
|
title = {OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models},
|
||||||
|
author = {Mengdi Jia and Zekun Qi and Shaochen Zhang and Wenyao Zhang and Xinqiang Yu and Jiawei He and He Wang and Li Yi},
|
||||||
|
journal = {arXiv preprint arXiv:2506.03135},
|
||||||
|
year = {2025}
|
||||||
|
}
|
||||||
|
```
|
||||||
29
adapter_config.json
Normal file
29
adapter_config.json
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
{
|
||||||
|
"alpha_pattern": {},
|
||||||
|
"auto_mapping": null,
|
||||||
|
"base_model_name_or_path": "UCSC-VLAA/VLAA-Thinker-Qwen2.5VL-3B",
|
||||||
|
"bias": "none",
|
||||||
|
"fan_in_fan_out": false,
|
||||||
|
"inference_mode": true,
|
||||||
|
"init_lora_weights": true,
|
||||||
|
"layer_replication": null,
|
||||||
|
"layers_pattern": null,
|
||||||
|
"layers_to_transform": null,
|
||||||
|
"loftq_config": {},
|
||||||
|
"lora_alpha": 256,
|
||||||
|
"lora_dropout": 0.05,
|
||||||
|
"megatron_config": null,
|
||||||
|
"megatron_core": "megatron.core",
|
||||||
|
"modules_to_save": null,
|
||||||
|
"peft_type": "LORA",
|
||||||
|
"r": 128,
|
||||||
|
"rank_pattern": {},
|
||||||
|
"revision": null,
|
||||||
|
"target_modules": [
|
||||||
|
"v_proj",
|
||||||
|
"q_proj"
|
||||||
|
],
|
||||||
|
"task_type": "CAUSAL_LM",
|
||||||
|
"use_dora": false,
|
||||||
|
"use_rslora": false
|
||||||
|
}
|
||||||
3
adapter_model.safetensors
Normal file
3
adapter_model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:c5289baa4e2b00730ff9967ed6f8c4a848ea4ec633729a91bfb7a5381af80beb
|
||||||
|
size 59002040
|
||||||
24
added_tokens.json
Normal file
24
added_tokens.json
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
{
|
||||||
|
"</tool_call>": 151658,
|
||||||
|
"<tool_call>": 151657,
|
||||||
|
"<|box_end|>": 151649,
|
||||||
|
"<|box_start|>": 151648,
|
||||||
|
"<|endoftext|>": 151643,
|
||||||
|
"<|file_sep|>": 151664,
|
||||||
|
"<|fim_middle|>": 151660,
|
||||||
|
"<|fim_pad|>": 151662,
|
||||||
|
"<|fim_prefix|>": 151659,
|
||||||
|
"<|fim_suffix|>": 151661,
|
||||||
|
"<|im_end|>": 151645,
|
||||||
|
"<|im_start|>": 151644,
|
||||||
|
"<|image_pad|>": 151655,
|
||||||
|
"<|object_ref_end|>": 151647,
|
||||||
|
"<|object_ref_start|>": 151646,
|
||||||
|
"<|quad_end|>": 151651,
|
||||||
|
"<|quad_start|>": 151650,
|
||||||
|
"<|repo_name|>": 151663,
|
||||||
|
"<|video_pad|>": 151656,
|
||||||
|
"<|vision_end|>": 151653,
|
||||||
|
"<|vision_pad|>": 151654,
|
||||||
|
"<|vision_start|>": 151652
|
||||||
|
}
|
||||||
3
assets/SpaceLLaVA.pdf
Normal file
3
assets/SpaceLLaVA.pdf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:08a1ee6adc1b3e6649ac509e9438287577976bb991c7360110ca5d8dace55ee3
|
||||||
|
size 3735716
|
||||||
3
chat_template.json
Normal file
3
chat_template.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}"
|
||||||
|
}
|
||||||
51
config.json
Normal file
51
config.json
Normal file
@@ -0,0 +1,51 @@
|
|||||||
|
{
|
||||||
|
"_name_or_path": "UCSC-VLAA/VLAA-Thinker-Qwen2.5VL-3B",
|
||||||
|
"architectures": [
|
||||||
|
"Qwen2_5_VLForConditionalGeneration"
|
||||||
|
],
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": 151643,
|
||||||
|
"eos_token_id": 151645,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 2048,
|
||||||
|
"image_token_id": 151655,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 11008,
|
||||||
|
"max_position_embeddings": 128000,
|
||||||
|
"max_window_layers": 70,
|
||||||
|
"model_type": "qwen2_5_vl",
|
||||||
|
"num_attention_heads": 16,
|
||||||
|
"num_hidden_layers": 36,
|
||||||
|
"num_key_value_heads": 2,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"rope_scaling": {
|
||||||
|
"mrope_section": [
|
||||||
|
16,
|
||||||
|
24,
|
||||||
|
24
|
||||||
|
],
|
||||||
|
"rope_type": "default",
|
||||||
|
"type": "default"
|
||||||
|
},
|
||||||
|
"rope_theta": 1000000.0,
|
||||||
|
"sliding_window": 32768,
|
||||||
|
"tie_word_embeddings": true,
|
||||||
|
"torch_dtype": "float16",
|
||||||
|
"transformers_version": "4.49.0",
|
||||||
|
"use_cache": false,
|
||||||
|
"use_sliding_window": false,
|
||||||
|
"video_token_id": 151656,
|
||||||
|
"vision_config": {
|
||||||
|
"hidden_size": 1280,
|
||||||
|
"in_chans": 3,
|
||||||
|
"model_type": "qwen2_5_vl",
|
||||||
|
"out_hidden_size": 2048,
|
||||||
|
"spatial_patch_size": 14,
|
||||||
|
"tokens_per_second": 2,
|
||||||
|
"torch_dtype": "float16"
|
||||||
|
},
|
||||||
|
"vision_end_token_id": 151653,
|
||||||
|
"vision_start_token_id": 151652,
|
||||||
|
"vision_token_id": 151654,
|
||||||
|
"vocab_size": 151936
|
||||||
|
}
|
||||||
13
generation_config.json
Normal file
13
generation_config.json
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"attn_implementation": "flash_attention_2",
|
||||||
|
"bos_token_id": 151643,
|
||||||
|
"do_sample": true,
|
||||||
|
"eos_token_id": [
|
||||||
|
151645,
|
||||||
|
151643
|
||||||
|
],
|
||||||
|
"pad_token_id": 151643,
|
||||||
|
"repetition_penalty": 1.05,
|
||||||
|
"temperature": 1e-06,
|
||||||
|
"transformers_version": "4.49.0"
|
||||||
|
}
|
||||||
3
gguf/spacethinker-qwen2.5VL-3B-F16.gguf
Normal file
3
gguf/spacethinker-qwen2.5VL-3B-F16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:82e066865117efc3410323d4c4bc3022846b2da449f4c7c28cdeaf28de0d0781
|
||||||
|
size 6178317632
|
||||||
3
gguf/spacethinker-qwen2.5VL-3B-q4_0.gguf
Normal file
3
gguf/spacethinker-qwen2.5VL-3B-q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:3abe0027365e6603b021a4671f1b91a6b82a0c7fdbba78b82ad895f3a72dc5f8
|
||||||
|
size 1822850368
|
||||||
3
gguf/spacethinker-qwen2.5VL-3B-q4_k_m.gguf
Normal file
3
gguf/spacethinker-qwen2.5VL-3B-q4_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:59a57561032bbe89397fd35b956e76f7d5e63587c4d38f98390250f2df5e8516
|
||||||
|
size 1929903424
|
||||||
3
gguf/spacethinker-qwen2.5vl-3b-vision.gguf
Normal file
3
gguf/spacethinker-qwen2.5vl-3b-vision.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:023b97965bd9212782e4485d2642b1ee8519721346735ef41e9a4cd61d7b67b7
|
||||||
|
size 2674769152
|
||||||
BIN
images/example_1.jpg
Normal file
BIN
images/example_1.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 90 KiB |
3
images/example_2.png
Normal file
3
images/example_2.png
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f92cb56983169f3a2ecc38e954f8ad72ad73f3ab8cfac60457fb3d03ecf82328
|
||||||
|
size 260943
|
||||||
3
images/example_3.png
Normal file
3
images/example_3.png
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e6d6eb7e4f82875fb0f282607e6088bc685bf0ce2a99450b2eeaef367dde84df
|
||||||
|
size 1064438
|
||||||
151388
merges.txt
Normal file
151388
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:529c9ad0489f963641373b7eaf092fbdaeacc43b6cd96f23898c4f886035b775
|
||||||
|
size 4997750128
|
||||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:fe22231694a8f501f09ef2794bf307db1c0114f54185256856d84f8adae446e5
|
||||||
|
size 2511586992
|
||||||
831
model.safetensors.index.json
Normal file
831
model.safetensors.index.json
Normal file
@@ -0,0 +1,831 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"total_size": 7509245952
|
||||||
|
},
|
||||||
|
"weight_map": {
|
||||||
|
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.32.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.norm.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.0.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.1.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.10.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.11.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.12.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.13.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.14.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.15.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.16.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.17.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.18.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.19.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.2.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.20.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.21.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.22.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.23.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.24.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.25.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.26.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.27.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.28.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.29.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.3.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.30.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.31.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.4.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.5.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.6.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.7.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.8.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.attn.proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.attn.proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.attn.qkv.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.attn.qkv.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.mlp.down_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.mlp.gate_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.mlp.up_proj.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.norm1.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.blocks.9.norm2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.merger.ln_q.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.merger.mlp.0.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.merger.mlp.0.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.merger.mlp.2.bias": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.merger.mlp.2.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"visual.patch_embed.proj.weight": "model-00001-of-00002.safetensors"
|
||||||
|
}
|
||||||
|
}
|
||||||
32
preprocessor_config.json
Normal file
32
preprocessor_config.json
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
{
|
||||||
|
"crop_size": null,
|
||||||
|
"default_to_square": true,
|
||||||
|
"do_center_crop": null,
|
||||||
|
"do_convert_rgb": true,
|
||||||
|
"do_normalize": true,
|
||||||
|
"do_rescale": true,
|
||||||
|
"do_resize": true,
|
||||||
|
"image_mean": [
|
||||||
|
0.48145466,
|
||||||
|
0.4578275,
|
||||||
|
0.40821073
|
||||||
|
],
|
||||||
|
"image_processor_type": "Qwen2VLImageProcessorFast",
|
||||||
|
"image_std": [
|
||||||
|
0.26862954,
|
||||||
|
0.26130258,
|
||||||
|
0.27577711
|
||||||
|
],
|
||||||
|
"max_pixels": 501760,
|
||||||
|
"merge_size": 2,
|
||||||
|
"min_pixels": 3136,
|
||||||
|
"patch_size": 14,
|
||||||
|
"processor_class": "Qwen2_5_VLProcessor",
|
||||||
|
"resample": 3,
|
||||||
|
"rescale_factor": 0.00392156862745098,
|
||||||
|
"size": {
|
||||||
|
"longest_edge": 1003520,
|
||||||
|
"shortest_edge": 3136
|
||||||
|
},
|
||||||
|
"temporal_patch_size": 2
|
||||||
|
}
|
||||||
31
special_tokens_map.json
Normal file
31
special_tokens_map.json
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
{
|
||||||
|
"additional_special_tokens": [
|
||||||
|
"<|im_start|>",
|
||||||
|
"<|im_end|>",
|
||||||
|
"<|object_ref_start|>",
|
||||||
|
"<|object_ref_end|>",
|
||||||
|
"<|box_start|>",
|
||||||
|
"<|box_end|>",
|
||||||
|
"<|quad_start|>",
|
||||||
|
"<|quad_end|>",
|
||||||
|
"<|vision_start|>",
|
||||||
|
"<|vision_end|>",
|
||||||
|
"<|vision_pad|>",
|
||||||
|
"<|image_pad|>",
|
||||||
|
"<|video_pad|>"
|
||||||
|
],
|
||||||
|
"eos_token": {
|
||||||
|
"content": "<|im_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
},
|
||||||
|
"pad_token": {
|
||||||
|
"content": "<|endoftext|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
}
|
||||||
|
}
|
||||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
|
||||||
|
size 11421896
|
||||||
212
tokenizer_config.json
Normal file
212
tokenizer_config.json
Normal file
@@ -0,0 +1,212 @@
|
|||||||
|
{
|
||||||
|
"add_bos_token": false,
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"added_tokens_decoder": {
|
||||||
|
"151643": {
|
||||||
|
"content": "<|endoftext|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151644": {
|
||||||
|
"content": "<|im_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151645": {
|
||||||
|
"content": "<|im_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151646": {
|
||||||
|
"content": "<|object_ref_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151647": {
|
||||||
|
"content": "<|object_ref_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151648": {
|
||||||
|
"content": "<|box_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151649": {
|
||||||
|
"content": "<|box_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151650": {
|
||||||
|
"content": "<|quad_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151651": {
|
||||||
|
"content": "<|quad_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151652": {
|
||||||
|
"content": "<|vision_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151653": {
|
||||||
|
"content": "<|vision_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151654": {
|
||||||
|
"content": "<|vision_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151655": {
|
||||||
|
"content": "<|image_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151656": {
|
||||||
|
"content": "<|video_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151657": {
|
||||||
|
"content": "<tool_call>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151658": {
|
||||||
|
"content": "</tool_call>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151659": {
|
||||||
|
"content": "<|fim_prefix|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151660": {
|
||||||
|
"content": "<|fim_middle|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151661": {
|
||||||
|
"content": "<|fim_suffix|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151662": {
|
||||||
|
"content": "<|fim_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151663": {
|
||||||
|
"content": "<|repo_name|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151664": {
|
||||||
|
"content": "<|file_sep|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additional_special_tokens": [
|
||||||
|
"<|im_start|>",
|
||||||
|
"<|im_end|>",
|
||||||
|
"<|object_ref_start|>",
|
||||||
|
"<|object_ref_end|>",
|
||||||
|
"<|box_start|>",
|
||||||
|
"<|box_end|>",
|
||||||
|
"<|quad_start|>",
|
||||||
|
"<|quad_end|>",
|
||||||
|
"<|vision_start|>",
|
||||||
|
"<|vision_end|>",
|
||||||
|
"<|vision_pad|>",
|
||||||
|
"<|image_pad|>",
|
||||||
|
"<|video_pad|>"
|
||||||
|
],
|
||||||
|
"bos_token": null,
|
||||||
|
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
|
||||||
|
"clean_up_tokenization_spaces": false,
|
||||||
|
"eos_token": "<|im_end|>",
|
||||||
|
"errors": "replace",
|
||||||
|
"extra_special_tokens": {},
|
||||||
|
"max_pixels": 501760,
|
||||||
|
"min_pixels": 3136,
|
||||||
|
"model_max_length": 131072,
|
||||||
|
"pad_token": "<|endoftext|>",
|
||||||
|
"processor_class": "Qwen2_5_VLProcessor",
|
||||||
|
"split_special_tokens": false,
|
||||||
|
"tokenizer_class": "Qwen2Tokenizer",
|
||||||
|
"unk_token": null,
|
||||||
|
"use_fast": true
|
||||||
|
}
|
||||||
243
train.py
Normal file
243
train.py
Normal file
@@ -0,0 +1,243 @@
|
|||||||
|
import re
|
||||||
|
import argparse
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import List
|
||||||
|
|
||||||
|
import torch
|
||||||
|
import wandb
|
||||||
|
from tqdm import tqdm
|
||||||
|
from PIL import Image
|
||||||
|
from datasets import load_dataset
|
||||||
|
from transformers import (
|
||||||
|
Qwen2_5_VLForConditionalGeneration,
|
||||||
|
AutoProcessor,
|
||||||
|
BitsAndBytesConfig,
|
||||||
|
)
|
||||||
|
from qwen_vl_utils import process_vision_info
|
||||||
|
from peft import LoraConfig, get_peft_model
|
||||||
|
from trl import SFTConfig, SFTTrainer
|
||||||
|
|
||||||
|
|
||||||
|
def extract_question(raw_text: str) -> str:
|
||||||
|
pattern = r"<\|start_header_id\|>user<\|end_header_id\|>\s*(.*?)\s*<\|eot_id\|>"
|
||||||
|
m = re.search(pattern, raw_text, re.DOTALL)
|
||||||
|
return m.group(1).strip() if m else raw_text.strip()
|
||||||
|
|
||||||
|
def format_data_spacethinker(sample):
|
||||||
|
system_message = {
|
||||||
|
"role": "system",
|
||||||
|
"content": [
|
||||||
|
{
|
||||||
|
"type": "text",
|
||||||
|
"text": (
|
||||||
|
"You are VL-Thinking U+1F914, a helpful assistant with excellent reasoning ability.\n"
|
||||||
|
"A user asks you a question, and you should try to solve it."
|
||||||
|
"You should first think about the reasoning process in the mind and then provides the user with the answer.\n"
|
||||||
|
"The reasoning process and answer are enclosed within <think></think> and <answer></answer> tags, respectively, i.e., <think> reasoning process here </think> <answer> answer here </answer>."
|
||||||
|
)
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
formatted = [system_message]
|
||||||
|
|
||||||
|
user_msg = {"role": "user", "content": []}
|
||||||
|
question = extract_question(sample.get("input", ""))
|
||||||
|
if question:
|
||||||
|
user_msg["content"].append({"type": "text", "text": question})
|
||||||
|
images = sample.get("images") or []
|
||||||
|
if images:
|
||||||
|
user_msg["content"].append({"type": "image", "image": images[0]})
|
||||||
|
formatted.append(user_msg)
|
||||||
|
|
||||||
|
if sample.get("output"):
|
||||||
|
formatted.append({
|
||||||
|
"role": "assistant",
|
||||||
|
"content": [{"type": "text", "text": sample["output"]}]
|
||||||
|
})
|
||||||
|
return formatted
|
||||||
|
|
||||||
|
|
||||||
|
def collate_fn(examples, processor):
|
||||||
|
# examples: list of formatted samples (list of message dicts)
|
||||||
|
texts = [processor.apply_chat_template(sample, tokenize=False) for sample in examples]
|
||||||
|
image_batches = [process_vision_info(sample)[0] for sample in examples]
|
||||||
|
batch = processor(text=texts, images=image_batches, return_tensors="pt", padding=True)
|
||||||
|
batch = {k: v.cpu() for k, v in batch.items()}
|
||||||
|
|
||||||
|
labels = batch["input_ids"].clone()
|
||||||
|
labels[labels == processor.tokenizer.pad_token_id] = -100
|
||||||
|
|
||||||
|
image_token_ids = (
|
||||||
|
[151652, 151653, 151655]
|
||||||
|
if hasattr(processor, "image_processor")
|
||||||
|
else [processor.tokenizer.convert_tokens_to_ids(processor.image_token)]
|
||||||
|
)
|
||||||
|
for tid in image_token_ids:
|
||||||
|
labels[labels == tid] = -100
|
||||||
|
|
||||||
|
batch["labels"] = labels
|
||||||
|
return batch
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TrainingConfig:
|
||||||
|
model_id: str = "UCSC-VLAA/VLAA-Thinker-Qwen2.5VL-3B"
|
||||||
|
dataset_id: str = "remyxai/SpaceThinker"
|
||||||
|
lora_r: int = 128
|
||||||
|
lora_alpha: int = 256
|
||||||
|
lora_dropout: float = 0.05
|
||||||
|
target_modules: List[str] = field(default_factory=lambda: ["q_proj", "v_proj"])
|
||||||
|
num_train_epochs: int = 3
|
||||||
|
train_batch_size: int = 1
|
||||||
|
eval_batch_size: int = 1
|
||||||
|
gradient_accumulation_steps: int = 8
|
||||||
|
learning_rate: float = 2e-5
|
||||||
|
warmup_ratio: float = 0.03
|
||||||
|
output_dir: str = "spacethinker-lora"
|
||||||
|
wandb_project: str = "spacethinker-lora"
|
||||||
|
wandb_run_name: str = "spacethinker_run"
|
||||||
|
|
||||||
|
|
||||||
|
def parse_args() -> TrainingConfig:
|
||||||
|
default_cfg = TrainingConfig()
|
||||||
|
parser = argparse.ArgumentParser(description="Train a VL Spacethinker model with LoRA")
|
||||||
|
parser.add_argument("--model_id", default=default_cfg.model_id)
|
||||||
|
parser.add_argument("--dataset_id", default=default_cfg.dataset_id)
|
||||||
|
parser.add_argument("--lora_r", type=int, default=default_cfg.lora_r)
|
||||||
|
parser.add_argument("--lora_alpha", type=int, default=default_cfg.lora_alpha)
|
||||||
|
parser.add_argument("--lora_dropout", type=float, default=default_cfg.lora_dropout)
|
||||||
|
parser.add_argument(
|
||||||
|
"--target_modules",
|
||||||
|
default=','.join(default_cfg.target_modules),
|
||||||
|
help="Comma-separated list of target modules for LoRA"
|
||||||
|
)
|
||||||
|
parser.add_argument("--num_train_epochs", type=int, default=default_cfg.num_train_epochs)
|
||||||
|
parser.add_argument("--train_batch_size", type=int, default=default_cfg.train_batch_size)
|
||||||
|
parser.add_argument("--eval_batch_size", type=int, default=default_cfg.eval_batch_size)
|
||||||
|
parser.add_argument(
|
||||||
|
"--gradient_accumulation_steps", type=int, default=default_cfg.gradient_accumulation_steps
|
||||||
|
)
|
||||||
|
parser.add_argument("--learning_rate", type=float, default=default_cfg.learning_rate)
|
||||||
|
parser.add_argument("--warmup_ratio", type=float, default=default_cfg.warmup_ratio)
|
||||||
|
parser.add_argument("--output_dir", default=default_cfg.output_dir)
|
||||||
|
parser.add_argument("--wandb_project", default=default_cfg.wandb_project)
|
||||||
|
parser.add_argument("--wandb_run_name", default=default_cfg.wandb_run_name)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
return TrainingConfig(
|
||||||
|
model_id=args.model_id,
|
||||||
|
dataset_id=args.dataset_id,
|
||||||
|
lora_r=args.lora_r,
|
||||||
|
lora_alpha=args.lora_alpha,
|
||||||
|
lora_dropout=args.lora_dropout,
|
||||||
|
target_modules=args.target_modules.split(","),
|
||||||
|
num_train_epochs=args.num_train_epochs,
|
||||||
|
train_batch_size=args.train_batch_size,
|
||||||
|
eval_batch_size=args.eval_batch_size,
|
||||||
|
gradient_accumulation_steps=args.gradient_accumulation_steps,
|
||||||
|
learning_rate=args.learning_rate,
|
||||||
|
warmup_ratio=args.warmup_ratio,
|
||||||
|
output_dir=args.output_dir,
|
||||||
|
wandb_project=args.wandb_project,
|
||||||
|
wandb_run_name=args.wandb_run_name,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_datasets(cfg: TrainingConfig):
|
||||||
|
print(f"Loading dataset: {cfg.dataset_id}…")
|
||||||
|
raw_train = load_dataset(cfg.dataset_id, split="train")
|
||||||
|
raw_eval = load_dataset(cfg.dataset_id, split="test")
|
||||||
|
|
||||||
|
print("Formatting train samples…")
|
||||||
|
train_ds = [format_data_spacethinker(s) for s in tqdm(raw_train, desc="Train")]
|
||||||
|
print("Formatting eval samples…")
|
||||||
|
eval_ds = [format_data_spacethinker(s) for s in tqdm(raw_eval, desc="Eval")]
|
||||||
|
|
||||||
|
return train_ds, eval_ds
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_model_and_optimizer(cfg: TrainingConfig):
|
||||||
|
bnb = BitsAndBytesConfig(
|
||||||
|
load_in_4bit=True,
|
||||||
|
bnb_4bit_use_double_quant=True,
|
||||||
|
bnb_4bit_quant_type="nf4",
|
||||||
|
bnb_4bit_compute_dtype=torch.bfloat16
|
||||||
|
)
|
||||||
|
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
|
||||||
|
cfg.model_id,
|
||||||
|
device_map="auto",
|
||||||
|
torch_dtype=torch.bfloat16,
|
||||||
|
quantization_config=bnb
|
||||||
|
)
|
||||||
|
processor = AutoProcessor.from_pretrained(cfg.model_id)
|
||||||
|
|
||||||
|
peft_cfg = LoraConfig(
|
||||||
|
r=cfg.lora_r,
|
||||||
|
lora_alpha=cfg.lora_alpha,
|
||||||
|
lora_dropout=cfg.lora_dropout,
|
||||||
|
bias="none",
|
||||||
|
target_modules=cfg.target_modules,
|
||||||
|
task_type="CAUSAL_LM",
|
||||||
|
)
|
||||||
|
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
||||||
|
peft_model = get_peft_model(model, peft_cfg).to(device)
|
||||||
|
peft_model.print_trainable_parameters()
|
||||||
|
return peft_model, processor, peft_cfg
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
cfg = parse_args()
|
||||||
|
train_ds, eval_ds = prepare_datasets(cfg)
|
||||||
|
model, processor, peft_cfg = prepare_model_and_optimizer(cfg)
|
||||||
|
|
||||||
|
sft_args = SFTConfig(
|
||||||
|
output_dir=cfg.output_dir,
|
||||||
|
num_train_epochs=cfg.num_train_epochs,
|
||||||
|
per_device_train_batch_size=cfg.train_batch_size,
|
||||||
|
per_device_eval_batch_size=cfg.eval_batch_size,
|
||||||
|
gradient_accumulation_steps=cfg.gradient_accumulation_steps,
|
||||||
|
gradient_checkpointing=True,
|
||||||
|
optim="adamw_torch_fused",
|
||||||
|
learning_rate=cfg.learning_rate,
|
||||||
|
lr_scheduler_type="constant",
|
||||||
|
logging_steps=10,
|
||||||
|
eval_steps=10,
|
||||||
|
eval_strategy="steps",
|
||||||
|
save_strategy="steps",
|
||||||
|
save_steps=20,
|
||||||
|
metric_for_best_model="eval_loss",
|
||||||
|
greater_is_better=False,
|
||||||
|
load_best_model_at_end=True,
|
||||||
|
bf16=True,
|
||||||
|
tf32=True,
|
||||||
|
max_grad_norm=0.3,
|
||||||
|
warmup_ratio=cfg.warmup_ratio,
|
||||||
|
gradient_checkpointing_kwargs={"use_reentrant": False},
|
||||||
|
push_to_hub=True,
|
||||||
|
report_to="wandb",
|
||||||
|
dataset_kwargs={"skip_prepare_dataset": True},
|
||||||
|
)
|
||||||
|
sft_args.remove_unused_columns = False
|
||||||
|
|
||||||
|
wandb.init(
|
||||||
|
project=cfg.wandb_project,
|
||||||
|
name=cfg.wandb_run_name,
|
||||||
|
config=sft_args,
|
||||||
|
)
|
||||||
|
|
||||||
|
trainer = SFTTrainer(
|
||||||
|
model=model,
|
||||||
|
args=sft_args,
|
||||||
|
train_dataset=train_ds,
|
||||||
|
eval_dataset=eval_ds,
|
||||||
|
data_collator=lambda ex: collate_fn(ex, processor),
|
||||||
|
peft_config=peft_cfg,
|
||||||
|
tokenizer=processor.tokenizer,
|
||||||
|
)
|
||||||
|
|
||||||
|
trainer.train()
|
||||||
|
trainer.save_model(cfg.output_dir)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user