初始化项目，由ModelHub XC社区提供模型

Model: IDEA-Research/Rex-Omni Source: Original Platform
2026-06-09 22:30:12 +08:00
commit 4a5b4255f2
17 changed files with 10457 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,167 @@
+---
+base_model:
+- Qwen/Qwen2.5-VL-3B-Instruct
+language:
+- en
+pipeline_tag: image-text-to-text
+tags:
+- vision
+- object-detection
+- multimodal
+- ocr
+- keypoint-detection
+- visual-prompting
+- open-set-detection
+- object-pointing
+library_name: transformers
+license: other
+---
+
+This model is **Rex-Omni**, a 3B-parameter Multimodal Large Language Model (MLLM) presented in the paper "[Detect Anything via Next Point Prediction](https://huggingface.co/papers/2510.12798)". It is compatible with the Hugging Face `transformers` library and is licensed under the [IDEA License 1.0](https://github.com/IDEA-Research/Rex-Omni/blob/main/LICENSE).
+
+<div align=center>
+  <img src="assets/logo.png" width=600 >
+</div>
+
+<h1 align="center">Detect Anything via Next Point Prediction</h1>
+
+<div align=center>
+
+<p align="center">
+  <a href="https://rex-omni.github.io/">
+    <img
+      src="https://img.shields.io/badge/RexOmni-Website-BADFDB?style=flat-square&logo=deno&logoColor=violet&color=BADFDB"
+      alt="RexThinker Website"
+    />
+  </a>
+  <a href="https://arxiv.org/abs/2510.12798">
+    <img
+      src="https://img.shields.io/badge/RexOmni-Paper-Red%25red?logo=arxiv&logoColor=red&color=yellow"
+      alt="RexThinker Paper on arXiv"
+    />
+  </a>
+  <a href="https://huggingface.co/IDEA-Research/Rex-Omni">
+    <img 
+        src="https://img.shields.io/badge/RexOmni-Weight-orange?logo=huggingface&logoColor=yellow" 
+        alt="RexThinker weight on Hugging Face"
+    />
+  </a>
+  <a href="https://huggingface.co/spaces/Mountchicken/Rex-Omni">
+    <img
+      src="https://img.shields.io/badge/RexOmni-Demo-orange?logo=huggingface&logoColor=yellow" 
+      alt="RexThinker Demo on Hugging Face"
+    />
+  </a>
+  <a href="https://github.com/IDEA-Research/Rex-Omni">
+    <img
+      src="https://img.shields.io/badge/GitHub-Code-blue?logo=github&logoColor=white"
+      alt="GitHub Code"
+    />
+  </a>
+</p>
+
+</div>
+
+> Rex-Omni is a 3B-parameter Multimodal Large Language Model (MLLM) that redefines object detection and a wide range of other visual perception tasks as a simple next-token prediction problem.
+
+<p align="center"><img src="assets/teaser.png" width="95%"></p>
+
+
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+conda create -n rexomni -m python=3.10
+pip install torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu124
+git clone https://github.com/IDEA-Research/Rex-Omni.git
+cd Rex-Omni
+pip install -v -e .
+```
+
+## 2. Quick Start: Using Rex-Omni for Detection
+
+```python
+from PIL import Image
+from rex_omni import RexOmniWrapper, RexOmniVisualize
+
+# Initialize model
+model = RexOmniWrapper(
+    model_path="IDEA-Research/Rex-Omni",
+    backend="transformers"  # or "vllm"
+)
+
+# Load image
+image = Image.open("your_image.jpg")
+
+# Object Detection
+results = model.inference(
+    images=image,
+    task="detection",
+    categories=["person", "car", "dog"]
+)
+
+result = results[0]
+
+# 4) Visualize
+vis = RexOmniVisualize(
+    image=image,
+    predictions=result["extracted_predictions"],
+    font_size=20,
+    draw_width=5,
+    show_labels=True,
+)
+vis.save("visualize.jpg")
+
+```
+
+## 3. Tutorials
+
+We provide a series of tutorials to help you get started with Rex-Omni.
+- [Detection Example](https://github.com/IDEA-Research/Rex-Omni/blob/master/tutorials/detection_example/_full_notebook.ipynb)
+- [Pointing Example](https://github.com/IDEA-Research/Rex-Omni/blob/master/tutorials/pointing_example/_full_tutorial.ipynb)
+- [OCR Example](https://github.com/IDEA-Research/Rex-Omni/blob/master/tutorials/ocr_example/_full_tutorial.ipynb)
+- [Keypointing Example](https://github.com/IDEA-Research/Rex-Omni/blob/master/tutorials/keypointing_example/_full_tutorial.ipynb)
+- [Visual Prompting Example](https://github.com/IDEA-Research/Rex-Omni/blob/master/tutorials/visual_prompting_example/_full_tutorial.ipynb)
+- [Batch Inference Example](https://github.com/IDEA-Research/Rex-Omni/blob/master/tutorials/other_example/batch_inference.py)
+
+
+## 📄 License
+
+Rex-Omni is licensed under the [IDEA License 1.0](LICENSE), Copyright (c) IDEA. All Rights Reserved. This model is based on Qwen, which is licensed under the [Qwen RESEARCH LICENSE AGREEMENT](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE), Copyright (c) Alibaba Cloud. All Rights Reserved.
+
+## 🔗 Links
+
+- 🏠 [Homepage](https://rex-omni.github.io/)
+- 🎮 [Demo](https://huggingface.co/spaces/Mountchicken/Rex-Omni)
+
+## 📧 Contact
+
+For questions and feedback, please contact us at:
+- Email: jiangqing@idea.edu.cn
+- GitHub Issues: [IDEA-Research/Rex-Omni](https://github.com/IDEA-Research/Rex-Omni/issues)
+
+## 7. Citation
+Rex-Omni comes from a series of prior works. If you’re interested, you can take a look.
+
+- [RexThinker](https://arxiv.org/abs/2506.04034)
+- [RexSeek](https://arxiv.org/abs/2503.08507)
+- [ChatRex](https://arxiv.org/abs/2411.18363)
+- [DINO-X](https://arxiv.org/abs/2411.14347)
+- [Grounidng DINO 1.5](https://arxiv.org/abs/2405.10300)
+- [T-Rex2](https://link.springer.com/chapter/10.1007/978-3-031-73414-4_3)
+- [T-Rex](https://arxiv.org/abs/2311.13596)
+
+
+```bibtex
+@misc{jiang2025detectpointprediction,
+      title={Detect Anything via Next Point Prediction}, 
+      author={Qing Jiang and Junan Huo and Xingyu Chen and Yuda Xiong and Zhaoyang Zeng and Yihao Chen and Tianhe Ren and Junzhi Yu and Lei Zhang},
+      year={2025},
+      eprint={2510.12798},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV},
+      url={https://arxiv.org/abs/2510.12798}, 
+}
+```