初始化项目,由ModelHub XC社区提供模型
Model: lmms-lab/MMSearch-R1-7B Source: Original Platform
This commit is contained in:
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.json filter=lfs diff=lfs merge=lfs -text
|
||||||
31
README.md
Normal file
31
README.md
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
---
|
||||||
|
license: apache-2.0
|
||||||
|
---
|
||||||
|
## MMSearch-R1-7B
|
||||||
|
|
||||||
|
### Introduction
|
||||||
|
MMSearch-R1-7B is a search-augmented LMM trained with end-to-end reinforcement learning, equipped with the ability to invoke multimodal search tools on demand. The model can dynamically decide whether to perform image or text search based on the question and integrate the retrieved external information into its reasoning process, enabling more accurate answers for knowledge-intensive VQA tasks. For more details on the training process and model evaluation, please refer to the [blog](https://www.lmms-lab.com/posts/mmsearch_r1/) or the [paper](https://arxiv.org/abs/2506.20670).
|
||||||
|
|
||||||
|
### Model Details
|
||||||
|
- Model name: MMSearch-R1-7B
|
||||||
|
- Architecture: Qwen2.5-VL-7B base model, fine-tuned with Reinforcement Learning (GRPO)
|
||||||
|
- Model type: Multimodal Large Language Model with Search-Augmentation
|
||||||
|
- Languages: English(primary), multilingual(partially)
|
||||||
|
- License: Apache license 2.0
|
||||||
|
- Paper: [MMSearch-R1: Incentivizing LMMs to Search](https://arxiv.org/abs/2506.20670)
|
||||||
|
- Code: [EvolvingLMMs-Lab/multimodal-search-r1](https://github.com/EvolvingLMMs-Lab/multimodal-search-r1)
|
||||||
|
|
||||||
|
### Training Details
|
||||||
|
- Dataset: [FVQA-train](https://huggingface.co/datasets/lmms-lab/FVQA)
|
||||||
|
- RL Framework: [veRL](https://github.com/volcengine/verl)
|
||||||
|
- GPUs: 32 * H100
|
||||||
|
|
||||||
|
### Citation
|
||||||
|
```
|
||||||
|
@article{wu2025mmsearch,
|
||||||
|
title={MMSearch-R1: Incentivizing LMMs to Search},
|
||||||
|
author={Wu, Jinming and Deng, Zihao and Li, Wei and Liu, Yiding and You, Bo and Li, Bo and Ma, Zejun and Liu, Ziwei},
|
||||||
|
journal={arXiv preprint arXiv:2506.20670},
|
||||||
|
year={2025}
|
||||||
|
}
|
||||||
|
```
|
||||||
BIN
added_tokens.json
(Stored with Git LFS)
Normal file
BIN
added_tokens.json
(Stored with Git LFS)
Normal file
Binary file not shown.
3
chat_template.json
Normal file
3
chat_template.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:94174d7176c52a7192f96fc34eb2cf23c7c2059d63cdbfadca1586ba89731fb7
|
||||||
|
size 1049
|
||||||
3
config.json
Normal file
3
config.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:fdd49987bd7322f7999049ee2655fc9a3033adbe47299c3ed6128da15d52ffae
|
||||||
|
size 1198
|
||||||
3
generation_config.json
Normal file
3
generation_config.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:27044301ebc98fb102106217c850461fb5e8eff3a2ab649bd9cdd3b0f3f203cb
|
||||||
|
size 121
|
||||||
151388
merges.txt
Normal file
151388
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00004.safetensors
Normal file
3
model-00001-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b7125d74c6b53a659f9bf5e298f4c47d61d0459b5aafdf55407d498fc01c595e
|
||||||
|
size 4966148280
|
||||||
3
model-00002-of-00004.safetensors
Normal file
3
model-00002-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:753ae893230f8c46cfa2f615a0bced172595a134601520380b4219e015fe6cb4
|
||||||
|
size 4871753776
|
||||||
3
model-00003-of-00004.safetensors
Normal file
3
model-00003-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:094f2d87f6c9d9f2c3ce473503e83b799db8b0567b929c5a2c0aff97a26fb53e
|
||||||
|
size 4967611072
|
||||||
3
model-00004-of-00004.safetensors
Normal file
3
model-00004-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ea48c16aa5a8203d6514a85171511b146d09e354e562b59faf2c9f5228884469
|
||||||
|
size 1778901376
|
||||||
3
model.safetensors.index.json
Normal file
3
model.safetensors.index.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8e547cfbdb979489eb08ab5994d0b5455639d038d8c7adc8894841bafe7795b5
|
||||||
|
size 57619
|
||||||
3
preprocessor_config.json
Normal file
3
preprocessor_config.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:549c158011407dfb750d9ec578047cf76f5bfe365cd0aa069a50137d3f98d9dd
|
||||||
|
size 575
|
||||||
BIN
special_tokens_map.json
(Stored with Git LFS)
Normal file
BIN
special_tokens_map.json
(Stored with Git LFS)
Normal file
Binary file not shown.
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
|
||||||
|
size 11421896
|
||||||
3
tokenizer_config.json
Normal file
3
tokenizer_config.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:c182cc5f590489eba7c5435864b0814fcd9256ef72933f8ccc142116974dbb01
|
||||||
|
size 7300
|
||||||
BIN
vocab.json
(Stored with Git LFS)
Normal file
BIN
vocab.json
(Stored with Git LFS)
Normal file
Binary file not shown.
Reference in New Issue
Block a user