S1-Parser/README.md

frameworks, license, tags, tasks

frameworks	license	tags	tasks
Pytorch	Apache License 2.0		image-text-to-text

📄 S1-Parser: Efficient Multimodal Document Parsing via Structure-Semantics Fusion

📑 Model Description

S1-Parser is a powerful and highly efficient multimodal text parsing tool anchored in a general Vision-Language Model (VLM), built through a two-stage training paradigm of Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL). It excels at accurately and efficiently parsing complex documents, with targeted optimization for critical metrics including formula syntax correctness, symbol integrity, and structural rationality; this design seamlessly balances parsing precision and inference efficiency across diverse document types.

Features

S1-Parser is built on a Transformers-driven vision-encoder-decoder architecture, tailored for multimodal text parsing tasks:

• 🧩 Supervised Fine-Tuning with task-oriented ([Parse Target: Scientific Equations]) to sharpen domain adaptation.
• 🎯 Multi-stage RL to refine, stabilize, and accelerate the learning process in strategic of behaviors.
• 📊 Benchmarked on Scientific Literature Dataset: SCI_LLM

The model supports both Unsloth FastVisionModel and Hugging Face Qwen2_5_VLForConditionalGeneration, enabling convenient loading and seamless integration with both the Unsloth and Transformers ecosystems.

🛠️ Usage

You can download the model via the following git clone or the ModelScope SDK

Download the model via the SDK

#Install the ModelScope SDK
pip install modelscope

#Download the model via the SDK
from modelscope import snapshot_download
model_dir = snapshot_download('xdcaxy2013/S1-Parser')

Download the model via the Git

#Download the model via the Git
git clone https://www.modelscope.cn/xdcaxy2013/S1-Parser.git

Model Loading via Unsloth FastVisionModel

import torch
from unsloth import FastVisionModel
from transformers import AutoProcessor, AutoTokenizer

# Core Loading Logic
model_path = "your-model-path"  
device = "cuda" if torch.cuda.is_available() else "cpu"
target_dtype = torch.bfloat16 if (device == "cuda" and torch.cuda.is_bf16_supported()) else torch.float32

# Load the Model, Tokenizer, and Processor
model, tokenizer = FastVisionModel.from_pretrained(
    model_name=model_path,
    max_seq_length=1024,
    load_in_4bit=False,
    load_in_8bit=False,
    device_map=device,
    low_cpu_mem_usage=True,
)
processor = AutoProcessor.from_pretrained(model_path, use_fast=False)

model = model.to(target_dtype).eval()# Inference Mode

Loading Based on Hugging Face Qwen2_5_VLForConditionalGeneration

import torch
from transformers import Qwen2_5_VLForConditionalGeneration, Qwen2VLProcessor, AutoTokenizer

# Core Loading Logic
model_path = "your-model-path" 
device = "cuda" if torch.cuda.is_available() else "cpu"
target_dtype = torch.bfloat16 if (device == "cuda" and torch.cuda.is_bf16_supported()) else torch.float32

# Load the Model, Tokenizer, and Processor
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
processor = Qwen2VLProcessor.from_pretrained(model_path, tokenizer=tokenizer, trust_remote_code=True)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_path,
    torch_dtype=target_dtype,
    device_map=device,
    low_cpu_mem_usage=True,
    trust_remote_code=True
)

model.eval() #Inference Mode

Our model supports the parsing of PDF and image formats (JPG, PNG, etc.), the following is an image data loading example program, compatible with JPG/PNG formats.

import torch
from PIL import Image

# Core data loading logic (image + prompt)
def load_data(image_path: str, prompt: str, processor) -> dict:
    # Load and preprocess image (JPG/PNG compatible)
    image = Image.open(image_path).convert("RGB")
    
    # Build model-compatible input template (text + image)
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "image_url", "image_url": {"url": "dummy"}}  # Dummy placeholder; actual image passed via processor
            ]
        }
    ]
    
    # Generate model input text with chat template
    text_input = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    
    # Process into model-acceptable tensor inputs
    inputs = processor(
        text=[text_input],
        images=[image],
        padding=True,
        return_tensors="pt",
        torch_dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float32
    ).to("cuda" if torch.cuda.is_available() else "cpu")
    
    return inputs

Please refer to our GitHub repository for detailed usage.

License

This model is licensed under the MIT License. This licensing and usage framework balances openness and accountability, making S1-Parser accessible for both academic exploration and practical application while protecting the rights of the original developers.

Citation

@software{s1parser2025,
  author    = {ScienceOne Team},
  title     = {S1-Parser Technical Report},
  year      = {2025},
  url       = {https://github.com/ScienceOne-AI/S1-Parser},
  publisher = {GitHub}
}

Acknowledgements

We would like to acknowledge several open-source projects that this model is built upon, as they have provided valuable foundations, inspiration for our work:

• Qwen2.5-VL
• olmOCR
• MonkeyOCR
• MistralOCR
• Dolphin

如果您是本模型的贡献者，我们邀请您根据模型贡献文档，及时完善模型卡片内容。