初始化项目,由ModelHub XC社区提供模型
Model: reducto/RolmOCR Source: Original Platform
This commit is contained in:
92
README.md
Normal file
92
README.md
Normal file
@@ -0,0 +1,92 @@
|
||||
---
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
datasets:
|
||||
- allenai/olmOCR-mix-0225
|
||||
base_model: Qwen/Qwen2.5-VL-7B-Instruct
|
||||
---
|
||||
|
||||
# RolmOCR by [Reducto AI](https://reducto.ai/)
|
||||
|
||||
Earlier this year, the [Allen Institute for AI](https://allenai.org/) released olmOCR, an open-source tool that performs document OCR using the Qwen2-VL-7B vision language model (VLM). We were excited to see a high-quality, openly available approach to parsing PDFs and other complex documents — and curious to explore what else might be possible using newer foundation models and some lightweight optimizations.
|
||||
|
||||
The result is **RolmOCR**, a drop-in alternative to olmOCR that’s faster, uses less memory, and still performs well on a variety of document types. We're releasing it under **Apache 2.0** for anyone to try out, explore, or build on.
|
||||
|
||||
This model is a fine-tuned version of [Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) on the full [allenai/olmOCR-mix-0225](https://huggingface.co/datasets/allenai/olmOCR-mix-0225) dataset.
|
||||
|
||||
## Key changes
|
||||
We made three notable changes:
|
||||
|
||||
1. **New Base Model**: We swapped in a more recent version of the existing model (Qwen2.5-VL-7B) as the foundation.
|
||||
|
||||
2. **No Metadata inputs**: Unlike the original, we don’t use metadata extracted from PDFs. This significantly reduces prompt length, which in turn lowers both processing time and VRAM usage — without hurting accuracy in most cases.
|
||||
|
||||
3. **Rotation of training data:** About 15% of the training data was rotated to enhance robustness to off-angle documents. We otherwise use the same training set.
|
||||
|
||||
## Usage
|
||||
|
||||
Host your model with vLLM:
|
||||
```bash
|
||||
export VLLM_USE_V1=1
|
||||
vllm serve reducto/RolmOCR
|
||||
```
|
||||
|
||||
Call the model via openai compatible server:
|
||||
```python
|
||||
# HOST YOUR OPENAI COMPATIBLE API WITH THE FOLLOWING COMMAND in VLLM:
|
||||
# export VLLM_USE_V1=1
|
||||
# vllm serve reducto/RolmOCR
|
||||
|
||||
from openai import OpenAI
|
||||
import base64
|
||||
|
||||
client = OpenAI(api_key="123", base_url="http://localhost:8000/v1")
|
||||
|
||||
model = "reducto/RolmOCR-7b"
|
||||
|
||||
def encode_image(image_path):
|
||||
with open(image_path, "rb") as image_file:
|
||||
return base64.b64encode(image_file.read()).decode("utf-8")
|
||||
|
||||
def ocr_page_with_rolm(img_base64):
|
||||
response = client.chat.completions.create(
|
||||
model=model,
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": f"data:image/png;base64,{img_base64}"},
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Return the plain text representation of this document as if you were reading it naturally.\n",
|
||||
},
|
||||
],
|
||||
}
|
||||
],
|
||||
temperature=0.2,
|
||||
max_tokens=4096
|
||||
)
|
||||
return response.choices[0].message.content
|
||||
|
||||
test_img_path = "path/to/image.png"
|
||||
img_base64 = encode_image(test_img_path)
|
||||
print(ocr_page_with_rolm(img_base64))
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- RolmOCR, like other VLM-based OCR solutions, still suffer from hallucination or dropping contents.
|
||||
- Unlike the [Reducto Parsing API](https://app.reducto.ai/), RolmOCR cannot output layout bounding boxes.
|
||||
- We have not evaluated the performance of any quantized versions.
|
||||
|
||||
## BibTex and citation info
|
||||
```
|
||||
@misc{RolmOCR,
|
||||
author = {Reducto AI},
|
||||
title = {RolmOCR: A Faster, Lighter Open Source OCR Model},
|
||||
year = {2025},
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user