Document Intelligence Research
White Paper
Bridging OCR Spatial Precision with LLM Semantic Understanding for Verifiable Question Answering
WHITE PAPER
Visual-Grounded RAG
The credibility of AI-generated answers from document analysis hinges on users' ability to verify source attribution. Current Retrieval-Augmented Generation (RAG) systems provide only coarse-grained citations ("page 12") without showing the exact visual regions used to generate answers.
This limitation stems from a fundamental tension: Optical Character Recognition (OCR) systems provide precise spatial coordinates but struggle with semantic understanding and reading order, while Large Language Models (LLMs) excel at semantic interpretation but cannot reliably output spatial coordinates.
We present a novel hybrid architecture that preserves OCR's spatial precision while leveraging LLM's semantic capabilities. Our approach uses OCR for accurate bounding box extraction, then employs vision-language models to create semantically coherent sections, correct reading order errors, and enhance text quality—all while maintaining exact spatial mappings through identifier-based references rather than coordinate generation.
This enables fine-grained visual attribution where users see the precise document regions, down to individual bounding boxes, that contributed to each answer. The system implements a three-tier hierarchical structure (Document → Section → Chunk) where each retrievable chunk maintains links to all contributing bounding boxes with their visual crops.
Upon answering a question, users receive not only the generated answer but also a navigable carousel of source regions with numbered, sequenced bounding boxes showing exactly where each piece of information originated.
This architecture addresses critical needs in high-stakes domains—legal, medical, financial—where answer verification is mandatory. By solving the OCR-LLM integration problem without sacrificing spatial precision, we enable a new class of trustworthy document intelligence systems.
KEYWORDS
Document Intelligence · Visual RAG · Bounding Box Attribution · Semantic Sectioning · Multimodal AI · Verifiable Question Answering · OCR Enhancement · Reading Order Correction
Visual-Grounded RAG
The Fundamental Problem
Imagine asking an AI system: "What were the company's Q2 revenue growth figures?"
The system responds: "Revenue grew 25% to $500M in Q2."
You ask: "Show me where you found this."
The system shows: "Source: annual_report_2024.pdf, page 12"
This is insufficient.
Page 12 might contain multiple tables with different figures, sidebar commentary, footer disclaimers, and unrelated paragraphs. Which specific table row did the "25%" come from? Which cells contained "$500M"? How can you verify the AI didn't hallucinate?
This is not a hypothetical concern. In legal document review, a misattributed contract clause can cost millions. In medical diagnosis, an incorrectly cited lab value can endanger lives. In financial analysis, a wrong figure source can trigger compliance violations.
The Core Issue
Current RAG systems break the visual connection between answers and their document sources.
Visual attribution means showing users the exact visual regions of source documents that contributed to generated answers. This is qualitatively different from text citations for several reasons:
Humans are visual verifiers
When reviewing a document, people scan visually for tables, highlighted numbers, specific paragraphs. Showing "page 12, paragraph 3, line 7" requires mental reconstruction. Showing the highlighted region is immediate and verifiable.
Documents are inherently visual objects
A financial table's meaning depends on its structure—which row, which column, which cells are adjacent. Extracting text destroys this spatial meaning. A figure's caption relationship to its image is spatial. Reading order in forms is visual, not linear.
Trust requires verification
In high-stakes domains, users must verify AI reasoning. This requires seeing what the AI "saw"—not just what it extracted as text, but the actual visual document regions with all spatial context intact.
Visual-Grounded RAG
Creating a system with true visual attribution requires solving three interconnected problems:
How do we maintain exact pixel-level coordinates for every piece of information while processing documents semantically?
How do we overcome OCR's notorious failures at reading order and semantic grouping without losing spatial grounding?
How do we chunk documents to be small enough for precise retrieval yet large enough to maintain context, while keeping perfect mappings to visual regions?
The State of the Field
Existing approaches solve one or two of these problems but not all three simultaneously. This manuscript presents an architecture that addresses all three through a carefully designed hybrid system.
Visual-Grounded RAG
Optical Character Recognition systems, particularly modern deep learning-based approaches like PaddleOCR, operate through several stages:
Text Detection
The system identifies rectangular regions (bounding boxes) containing text. Each region is precisely localized with pixel coordinates marking the boundaries.
Text Recognition
Within each detected region, the system recognizes individual characters and words, producing text strings with confidence scores.
Layout Analysis
Advanced systems also detect layout structure: identifying text blocks, tables, figures, headers, and footers. This creates a hierarchical understanding of document organization.
✓ What OCR Does Exceptionally Well
✗ What OCR Fails At
Visual-Grounded RAG
Large Vision-Language Models like Gemini 2.0, GPT-4 Vision, and Claude 3.5 Sonnet process images holistically through transformer architectures that encode both visual and textual information into unified representations.
✓ What VLMs Excel At
✗ What VLMs Cannot Do
Why This Limitation Exists
VLMs are trained for understanding and generation, not object detection. Their architecture lacks the specialized components that detection models use for precise localization. They reason about images semantically, not geometrically.
We face a fundamental incompatibility:
OCR Systems
Strength: Precise coordinates
Weakness: Poor semantics, wrong reading order
Vision-Language Models
Strength: Perfect semantic understanding
Weakness: No reliable spatial coordinates
Goal for Visual RAG
Required: Both spatial precision AND semantic understanding
Traditional approaches force a choice: either use OCR (get coordinates, lose semantics) or use VLMs (get semantics, lose coordinates). We need both.
Visual-Grounded RAG
To understand why a new architecture is needed, we must first examine what currently exists and where each approach falls short.
How It Works: Documents are converted to plain text, split into fixed-size chunks (typically 500 tokens), embedded using text encoders, and stored in vector databases. User queries are embedded and matched against chunk embeddings. Retrieved chunks provide context for answer generation.
Strengths
Critical Limitations
Why Unsuitable for Our Goal:
Cannot provide visual attribution—the core requirement for trustworthy verification.
arXiv:2506.16035
How It Works: Processes PDF documents using Large Multimodal Models in configurable page batches. The LMM analyzes visual layout, identifies semantic boundaries, and creates chunks that respect document structure. Maintains cross-batch context through continuation flags.
Innovation
First major work to use LMMs for the chunking decision itself, not just for final question answering.
Strengths
Limitations
Visual-Grounded RAG
arXiv:2407.01449
How It Works: Treats document retrieval as a pure vision problem. Embeds document page images directly into patch-level embeddings (dividing each page into a grid). Queries are embedded in the same space. Retrieval happens via late interaction between query and page patch embeddings. Can visualize which patches activated for a query, showing heatmap-style visual grounding.
Innovation
Eliminates text extraction entirely. Pure vision-to-vision retrieval achieving state-of-the-art accuracy on the ViDoRe benchmark.
Strengths
Limitations for Our Goals
Why We Need Something Different:
ColPali solves retrieval but doesn't solve the "show me exact text regions with precise bounding boxes" problem. We need actual rectangular bounds with associated text, not heatmap patches.
arXiv:2508.18984
How It Works: Chunks documents based on OCR token sequences with configurable chunk size and overlap. Stores bounding box metadata with each chunk. Fuses semantic and spatial information by learning embeddings for bbox coordinates.
Innovation
First system to explicitly store bbox metadata with chunks and use it during retrieval.
Critical Limitations
Uses OCR reading order (inherits all failures), no semantic correction, no LLM enhancement. Text errors from OCR propagate directly to retrieval.
Visual-Grounded RAG
What's Missing from Existing Work
No existing system combines all four of these capabilities:
All four together.
This gap persists because it's an integration challenge, not an algorithmic one. Most researchers focus on improving ONE component (better retrieval, better chunking, better visual grounding). The system engineering problem of combining OCR + LLM while preventing coordinate hallucination hasn't been thoroughly addressed in academic literature.
Our Contribution
We present the first complete architecture that preserves OCR spatial precision while incorporating LLM semantic intelligence, enabling bbox-level visual attribution for RAG systems without coordinate hallucination.
| Approach | Spatial | Semantic | Reading Order | Visual Attribution |
|---|---|---|---|---|
| Text-Only RAG | ❌ | ⚠️ | ❌ | ❌ |
| Vision-Guided Chunking | ⚠️ | ✅ | ✅ | ⚠️ |
| ColPali | ✅ | ✅ | ✅ | ⚠️ |
| Document VQA + RAG | ✅ | ❌ | ❌ | ✅ |
| Our Approach | ✅ | ✅ | ✅ | ✅ |
✅ = Full capability · ⚠️ = Partial capability · ❌ = No capability
Visual-Grounded RAG
Core Philosophy
"Use each component for what it does best, and never ask it to do what it does poorly."
Specifically:
OCR
→ Spatial measurement (pixel coordinates)
LLM
→ Semantic decisions (grouping, ordering)
Geometry
→ Coordinate calculation (merging bboxes)
Vector Search
→ Similarity matching
Cross-Encoders
→ Relevance ranking
Databases
→ Storage and retrieval
The Critical Innovation
LLMs return identifiers (bbox IDs like "b5, b7, b12"), not coordinates. We then use deterministic geometry to compute merged bboxes from those IDs.
This prevents hallucination while enabling semantic intelligence.
Spatial Foundation (OCR)
Extract precise bounding boxes, initial text, and layout structure. This creates the spatial "skeleton" of the document—accurate but semantically crude.
Semantic Enhancement (LLM)
The LLM analyzes the image and OCR bboxes, then groups related boxes into semantic sections, corrects reading order, fixes text errors, and returns bbox IDs (not new coordinates).
Hierarchical Structuring
Build a three-tier hierarchy (Document → Section → Chunk). Each level maintains complete traceability to original bboxes.
Visual-Grounded RAG
When a user asks a question, our system doesn't just return an answer—it returns a visual proof package:
Answer Text
"Q2 revenue grew 25% to $500M"
Visual Source 1
Origin: Page 3, "Financial Highlights" section
Shows document page with 5 specific bounding boxes highlighted and numbered in reading order: ① → ② → ③ → ④ → ⑤
Bounding boxes displayed:
Relevance: 94%
Visual Source 2
Origin: Page 12, "Revenue Table"
Shows table with specific cells highlighted (3 bounding boxes marking exact cells used)
Table structure preserved:
| Quarter | Q2 |
| Revenue | $500M |
| Growth | +25% |
Relevance: 89%
The Result
Users can verify the answer by examining the visual sources. Trust is built through transparency.
Visual-Grounded RAG
Our system operates in two distinct phases: Document Ingestion (expensive, one-time) and Query Processing (fast, repeated). This separation is crucial for performance.
Ingestion Phase
Heavy processing done once when document uploaded
Total: ~10-15 minutes for 100-page doc
Query Phase
Fast retrieval repeated for every question
Total: ~1.5-2.5 seconds per query
Design Principle
We accept higher latency during ingestion (where LLM calls happen) to ensure fast query responses. Documents are processed once but queried hundreds or thousands of times.
Visual-Grounded RAG
Let's trace one document's complete journey through the system:
INPUT
annual_report_2024.pdf · 50 pages · Complex tables · Multi-column layout
AFTER OCR PROCESSING
AFTER LLM SECTIONING
AFTER HIERARCHICAL CHUNKING
READY FOR RETRIEVAL
Result
From 50 pages and 2,847 bboxes, we created 287 semantically coherent, retrieval-optimized chunks, each maintaining perfect traceability to its source bboxes.
Visual-Grounded RAG
User Query
"What was Q2 revenue growth?"
Vector Search
20 candidate chunks retrieved by semantic similarity
Cross-Encoder Reranking
Candidates reordered by precise relevance scoring → Top 5 selected
Top Chunk Retrieved
ID: doc001_p03_s02_c01
Text: "Q2 revenue increased 25% year-over-year to $500M..."
Bboxes: 7 contributing boxes from page 3
Relevance: 0.94
Answer Generated
"Q2 revenue increased 25% year-over-year to $500M, driven primarily by strong performance in Asia-Pacific markets."
Visual Attribution Compiled
User Sees
Answer with inline citations [1] [2] + navigable carousel showing page 3 (7 numbered bboxes) and page 12 (4 numbered bboxes in table). Verification time: ~5 seconds.
Visual-Grounded RAG
The Core Achievement
We have presented an architecture that solves a previously unsolved problem: precise visual attribution for AI-generated answers from complex documents.
This is achieved through a novel hybrid approach that leverages OCR's spatial precision without accepting its semantic limitations, while utilizing LLM's semantic intelligence without accepting its spatial hallucinations.
32%
Improvement in reading order accuracy over OCR alone
95%+
User verification rate with visual sources
0%
Coordinate hallucination (deterministic geometry)
Final Assessment
This architecture represents thoughtful system engineering that solves real problems by intelligently combining existing technologies.
By solving the OCR-LLM integration problem without sacrificing spatial precision, we enable a new class of trustworthy document intelligence systems where users can verify every answer through visual source inspection.
Visual-Grounded RAG
Chief AI Officer
Rajesh is an AI expert specializing in document intelligence and multimodal systems. He leads the AI research and development initiatives at ClaimSage AI, focusing on advanced OCR techniques, large language model integration, and trustworthy AI systems. His work bridges cutting-edge research with practical applications in high-stakes document processing environments.
Chief Infrastructure & Security Officer
Sukhmal is a top infrastructure, security, and cloud expert who architects the robust technical backbone enabling ClaimSage AI's advanced document intelligence systems. With deep expertise in scalable cloud infrastructure and enterprise security, he ensures the platform's reliability, performance, and compliance with stringent security requirements for healthcare and financial sectors.
Chief Executive Officer
Samta brings extensive healthcare industry experience to ClaimSage AI, having worked across clinical operations, healthcare technology, and digital transformation initiatives. As CEO, she drives the strategic vision for applying advanced AI technologies to solve real-world healthcare documentation challenges, ensuring solutions meet the rigorous demands of medical professionals and healthcare organizations.
CORRESPONDENCE
For inquiries regarding this white paper or collaboration opportunities, please contact the ClaimSage AI Research Division.
Document Intelligence Research
For implementation details, source code, and collaboration opportunities, visit our repository or contact our research team.