From 4738fd54724d74b4e769ac9303fbf18e3e72fd9b Mon Sep 17 00:00:00 2001 From: Odunayo Ogundepo Date: Sun, 25 Jan 2026 11:41:24 +0000 Subject: [PATCH] Update README.md --- README.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 0251fd9..d89c121 100644 --- a/README.md +++ b/README.md @@ -78,23 +78,26 @@ While improved performance on African languages was our priority, KarantaOCR **m ## Evaluation KarantaOCR is evaluated on the OLMOocr benchmark using pass-rate accuracy. Scores are reported as averages across JSONL files with 95% confidence intervals. +In addition to OLMOocr benchmark, we also create a KarantaOCR-Bench, which focuses specifically on testing OCR extraction on special characters and diacritics. ### Results -- KarantaOCR-Bench -... +| JSONL File | KarantaOCR (3B) | RoLMOCR (7B) | NanoNetsOCR-2 (3B) | OLMOCR-2 (7B) | +| --------------- | ---------- | -------- | ------------- | -------- | +| Diacritics & Special Characters | **61.6** | 53.6 | 55.1 | 55.5 ### Results -- OlmoOCR-Bench | JSONL File | KarantaOCR (3B) | RoLMOCR (7B) | NanoNetsOCR-2 (3B) | OLMOCR-1 (7B) | OLMOCR-2 (7B) | Mistral OCR API | DeepSeek-OCR (3B) | | --------------- | ---------- | -------- | ------------- | -------- | -------- |-------- |-------- | -| arxiv_math | 74.2 | **76.8** | 73.7 | 63.3 | 83.0 | 77.2 | 77.2 | -| baseline | **99.4** | 97.9 | **99.5** | 97.9 | 99.7 | 99.4 | 99.8 | -| headers_footers | **95.3** | 94.1 | 32.8 | 93.4 | 96.1 | 93.6 | 96.1 | -| long_tiny_text | 72.2 | 61.3 | **92.1** | 54.8 | 81.9 | 77.1 | 79.4 | -| multi_column | 75.6 | 70.0 | **82.5** | 67.6 | 83.7 | 71.3 | 66.4 | +| arxiv_math | 74.2 | 76.8 | 73.7 | 63.3 | 83.0 | 77.2 | 77.2 | +| baseline | 99.4 | 97.9 | 99.5 | 97.9 | 99.7 | 99.4 | 99.8 | +| headers_footers | 95.3 | 94.1 | 32.8 | 93.4 | 96.1 | 93.6 | 96.1 | +| long_tiny_text | 72.2 | 61.3 | 92.1 | 54.8 | 81.9 | 77.1 | 79.4 | +| multi_column | 75.6 | 70.0 | 82.5 | 67.6 | 83.7 | 71.3 | 66.4 | | old_scans | 41.3 | 42.4 | 41.4 | 38.6 | 47.7 | 29.3 | 33.3 | -| old_scans_math | 70.3 | **80.1** | 44.1 | 67.5 | 82.3 | 67.5 | 73.6 | -| table_tests | 64.3 | 72.2 | **84.2** | 62.3 | 84.9 | 60.6 | 80.2 | +| old_scans_math | 70.3 | 80.1 | 44.1 | 67.5 | 82.3 | 67.5 | 73.6 | +| table_tests | 64.3 | 72.2 | 84.2 | 62.3 | 84.9 | 60.6 | 80.2 | | Average | 74.1% | 74.4% | 68.8% | 68.2% | 82.4% | 72.0 | 75.7 |