---
license: cc-by-4.0
language:
- en
base_model: Qwen/Qwen3-VL-4B-Instruct
library_name: transformers
tags:
- svg
- vision-language
- reinforcement-learning
- grpo
- figure-understanding
- vector-graphics
---

# VFIG-4B

This repository contains the merged RL-trained checkpoint for **VFIG-4B**, a 4B vision-language model for converting complex scientific and technical figures into clean, editable SVG code.

- 📄 [Paper](https://arxiv.org/abs/2603.24575)

## Model Details

| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3-VL-4B-Instruct |
| Training | SFT + GRPO-based RL with rendering-aware visual rewards |
| Architecture | LoRA on language model; vision encoder and projector frozen |
| Parameters | 4B |
| Precision | BF16 |

## Intended Use

Given an input image of a scientific or technical figure, the model generates SVG code that reconstructs the figure as a scalable, editable vector graphic.

## Quick Start

```bash
pip install transformers torch accelerate pillow
```

```python
import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText

model_name = "XunmeiLiu/VFIG-4B"

processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    trust_remote_code=True,
)
model.eval()

def figure_to_svg(image_path: str) -> str:
    img = Image.open(image_path).convert("RGB")

    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image"},
                {"type": "text", "text": "Convert this figure into valid SVG code."},
            ],
        }
    ]

    chat_input = processor.tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    inputs = processor(text=[chat_input], images=[img], return_tensors="pt").to("cuda")

    with torch.no_grad():
        output_ids = model.generate(**inputs, max_new_tokens=8192, do_sample=False)

    decoded = processor.tokenizer.decode(output_ids[0], skip_special_tokens=True)

    if "<svg" in decoded:
        decoded = decoded[decoded.find("<svg"):]
    if "</svg>" in decoded:
        decoded = decoded[: decoded.find("</svg>") + len("</svg>")]
    return decoded.strip()

# Download the example image
import urllib.request
urllib.request.urlretrieve(
    "https://huggingface.co/XunmeiLiu/VFIG-4B/resolve/main/simple_diagram.png",
    "simple_diagram.png"
)

svg_code = figure_to_svg("simple_diagram.png")
print(svg_code)
```

## License

This repository uses the `cc-by-4.0` license. Please also ensure that redistribution is compatible with the license of the underlying base model.

## Citation

If you use VFIG-4B in your research, please cite:

```bibtex
@misc{he2026vfigvectorizingcomplexfigures,
      title={VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models}, 
      author={Qijia He and Xunmei Liu and Hammaad Memon and Ziang Li and Zixian Ma and Jaemin Cho and Jason Ren and Daniel S Weld and Ranjay Krishna},
      year={2026},
      eprint={2603.24575},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2603.24575}, 
}