Document Parsing for AI Agents: ParseBench, dots.mocr, and the New OCR Stack

AI agents do not need a prettier OCR transcript. They need a document interface they can trust. That means page images, text, tables, charts, layout blocks, bounding boxes, and enough provenance to prove where each answer came from. The recent wave of vision-language document research, especially ParseBench and dots.mocr, is useful because it moves the conversation from “can this model read the page?” to “can this system preserve the parts an agent needs to act?”

That is a different bar. A support bot can survive a loose summary. A financial agent, compliance agent, or coding agent touching PDFs needs stable facts, typed fields, page citations, and a way to recover when the parser is wrong.

Table of Contents

• Why agent-facing document parsing is different
• What ParseBench changed
• What dots.mocr adds to the conversation
• Build an agent-ready document pipeline
• Evaluate parsers for agent work
• Implementation shape for agent tools
• Where okraPDF fits

Why agent-facing document parsing is different

OCR text is not the interface

Plain OCR text is a lossy debug artifact. It can help a human skim a scanned page, but it is not enough for an agent that needs to calculate, cite, compare, or write data into another system.

A useful parser has to keep the document as a set of artifacts:

• rendered page images
• text spans
• layout regions
• tables and cells
• charts and figures
• extracted key-value fields
• citations back to page regions
• raw vendor output for later review

That artifact set is what lets an agent answer a question and show its work.

The difference shows up quickly in production. An agent asked to “find the renewal term” may need a paragraph. An agent asked to “reconcile the invoice total” needs table structure, currency, subtotals, and row labels. An agent asked to “explain the chart on page 12” needs a visual region, not just the caption that happened to be nearby.

This is why “OCR accuracy” is the wrong headline metric for agent work. The useful metric is whether the parser produced enough stable structure for the next tool call to be deterministic.

Agents need verifiable structures

When an agent extracts “revenue was 88,268”, the system needs to know more than the string. It should know the page, table, row label, column label, bbox, parser, model version, and the original visual evidence.

That turns a parse result into a contract:

{
  "field": "revenue",
  "value": 88268,
  "unit": "USD millions",
  "page": 7,
  "evidence": {
    "node_id": "node_01j...",
    "bbox": [0.12, 0.44, 0.88, 0.58],
    "source_text": "Revenues 88,268",
    "parser": "vision-layout-v3"
  }
}

Without that evidence, an agent can only ask the model to be careful. That is not an engineering control.

The better pattern is to make the evidence boring and explicit. Every field that could trigger an action should carry a reference to the source region. Every table-derived value should carry row and column context. Every generated answer should be able to show the page that supports it.

That sounds like extra metadata, but it is what lets an agent recover. If the field is suspicious, the agent can fetch the page crop. If two parsers disagree, the agent can compare sources. If a human corrects a value, the system can attach that correction to the exact fixture that failed.

The parser output becomes product surface

The parser is no longer buried behind a search index. It becomes the product surface agents call directly.

That is why document parsing is converging with API design. If the output is unstable, every agent prompt becomes fragile. If the output is structured, cited, and queryable, an agent can use normal software patterns: filter nodes, join cells, validate schema, retry weak pages, and ask a human only when the evidence is ambiguous.

The agent does not have to know every PDF trick. It needs a small, dependable set of document operations: inspect, query, extract, verify, and display. Good parsing turns the PDF from an opaque file into that operation surface.

What ParseBench changed

It evaluates more than text extraction

ParseBench is useful because it treats document parsing as a multi-capability problem. The paper, dataset, and evaluation code focus on the pieces that make real PDFs hard: tables, charts, semantic formatting, visual grounding, and faithfulness.

That matters for agents because failures are not evenly distributed. A parser can look excellent on paragraphs and still fail at the exact line item, chart label, or merged table header an agent needs.

This is the right benchmark shape for agent tooling. It breaks the task into capabilities that product teams can reason about. A document agent that handles contracts may care most about layout order and citations. A finance agent may care most about table cells, units, and chart values. A support agent may care about headings, lists, and procedural steps.

Enterprise PDFs are the right stress test

The dataset direction also matters. Enterprise PDFs are not clean academic OCR examples. They include annual reports, invoices, decks, manuals, scanned pages, footnotes, nested tables, and visual hierarchy that was designed for humans rather than machines.

That is the environment where agents actually operate. If the benchmark only rewards reading clean text, it overstates agent readiness.

The important detail is that enterprise documents mix clean and hostile regions on the same page. One section may have selectable text. The next may use a rasterized chart. A table may be born-digital but visually split across pages. A footnote may change the meaning of a number. Agents need a parser that can preserve those transitions instead of flattening the whole page into one markdown stream.

Capability	Human reader expectation	Agent requirement
Paragraph text	Read the section	Retrieve the right span with page evidence
Tables	Understand rows and columns visually	Preserve cell values, headers, units, and order
Charts	Infer the trend	Extract data or produce a cited visual summary
Layout	Follow the page flow	Keep regions and reading order stable
Formatting	Notice emphasis and hierarchy	Preserve headings, lists, captions, and footnotes

No single parser wins every job

The practical lesson is not “one VLM won.” It is that parser choice has to become task-aware.

A cheap text-layer parser may be enough for born-digital contracts. A VLM parser may be necessary for scanned invoices, complex tables, and chart-heavy reports. A high-resolution layout model may be useful on a few pages and wasteful on the rest.

For agent systems, the question becomes:

• Which pages need visual parsing?
• Which fields need bbox evidence?
• Which tables need deterministic validation?
• Which parser is good enough for this document class?
• Which failures should block automation?

That is exactly the evaluation loop ParseBench pushes teams toward.

It also discourages cargo-culting model rankings. A leaderboard is a starting point for parser selection, not a production policy. The production policy should say when to use the cheap parser, when to escalate to a VLM, when to require bbox evidence, and when to block the agent from taking an action.

What dots.mocr adds to the conversation

Charts and graphics are first-class document content

dots.mocr is interesting because it takes multimodal document parsing beyond OCR text. The paper frames the job as parsing text, tables, and graphics from complex documents, including outputs that can represent visual structure rather than flattening everything into prose.

That direction is important for agents. A quarterly report chart is not decorative. A process diagram in a manual is not decorative. An org chart in a compliance packet is not decorative. These objects carry operational information.

For many business documents, the most important claim is visual. A risk heatmap, timeline, waterfall chart, or table embedded in a slide can carry the answer while the surrounding prose only gestures at it. If the parser ignores the visual object, the agent sees the least useful part of the page.

Structured visual output changes what agents can do

If a model can turn a chart into an inspectable representation, an agent can do more than summarize it. It can compare values, detect trends, generate a recreated graphic, or cite the exact page region where the visual fact came from.

The dots.mocr repository points toward that kind of parser output. The broader dots.ocr line reinforces the same idea: OCR and document parsing are becoming multimodal layout systems, not just text recognition systems.

For a document agent, that unlocks a cleaner split:

• the VLM reads difficult visual regions
• deterministic code validates tables and values
• the agent reasons over typed, cited artifacts
• the UI shows the source region when confidence is low

That split is healthier than asking one large model to read the whole file and produce a final answer. The parser can specialize in visual recovery. The database can specialize in retrieval. Deterministic code can specialize in validation. The agent can specialize in planning, tool use, and communicating the result.

The hard part is still reliability

The research direction is strong, but production systems still need guardrails.

VLM parsers can hallucinate table structure, over-read decorative labels, lose units, or normalize values too aggressively. They may also disagree with text-layer extraction on born-digital pages. That disagreement is not noise. It is a signal the system should preserve.

A good system treats parser disagreement like test output. It keeps the conflicting values, records which parser produced each one, and routes the disputed field into verification. That is slower than a single markdown answer, but it is much safer for workflows where an agent might update a CRM, file a report, send a customer reply, or trigger a payment review.

An agent-ready parser should keep multiple facets side by side instead of collapsing them too early:

Facet	Best use	Failure mode
Text layer	Fast born-digital text	misses scanned pages and visual context
OCR text	scanned paragraphs	loses table semantics and reading order
Layout nodes	headings, regions, cells	needs normalization and stable IDs
VLM markdown	rich page understanding	may smooth over exact evidence
Bbox evidence	verification and UI review	expensive if generated for every page

Build an agent-ready document pipeline

Ingest once and preserve the source

The first mistake is treating parsing as a one-shot text conversion. Ingest should create a durable document identity and preserve the source PDF, page images, and raw parser payloads.

For agents, stable file access matters. If the same document can be uploaded twice and produce unrelated IDs, every cache, citation, and audit trail becomes harder. A hosted document surface from /host or a durable extraction surface from /extract should be a starting point, not an afterthought.

The document identity becomes the anchor for every later operation. It ties together user permissions, parser versions, page assets, extracted nodes, saved eval cases, and human corrections. Without that anchor, each agent run becomes a disconnected experiment.

Parse into multiple facets

Think of parsing as facet generation. Each facet answers a different question.

{
  "document_id": "doc_01j...",
  "facets": {
    "page_images": "ready",
    "text_layer": "ready",
    "layout_nodes": "ready",
    "tables": "ready",
    "vision_markdown": "ready",
    "bbox_citations": "partial"
  }
}

That makes an agent workflow more robust. A cheap query can start with text. A table task can jump to cells. A chart task can ask for visual parsing. A verification task can request page images and bboxes.

Facet-based parsing also helps cost control. You do not need a high-end VLM pass over every page of every document. You need enough metadata to decide which pages deserve escalation. That can mean text-layer first, layout second, visual parsing only for pages with tables, charts, scans, or failed checks.

Store evidence with every field

The useful output is not just JSON. It is JSON plus evidence.

type ExtractedField = {
  name: string;
  value: string | number | boolean | null;
  confidence?: number;
  page: number;
  nodeId?: string;
  bbox?: [number, number, number, number];
  sourceText?: string;
  parser: string;
};

That shape gives the agent a path back to the page. It also lets your app decide when to trust automation and when to ask for review.

This is where document parsing and governance start to meet. Access control, retention, redaction, and audit logs all become easier when the extracted data keeps its source pointer. If a field is sensitive, you can find the exact page region. If a user asks why an agent made a decision, you can show the evidence instead of replaying a prompt transcript.

Evaluate parsers for agent work

Build page-level fixtures

Agent evaluation should not start with “summarize this PDF.” It should start with page-level fixtures that encode the hard parts of the document class.

Good fixtures look like this:

{
  "document": "sample-10k.pdf",
  "page": 42,
  "checks": [
    {
      "name": "revenue_cell",
      "type": "table_cell",
      "row": "Revenue",
      "column": "2025",
      "expected": "88,268"
    },
    {
      "name": "risk_heading",
      "type": "heading_exists",
      "expected": "Liquidity and Capital Resources"
    }
  ]
}

That is the practical lesson from ParseBench for product teams: create checks that represent the work your agent actually needs to do.

The fixtures should be small enough to run often. A ten-page vendor packet with five hard checks is more useful than a thousand-page corpus nobody can interpret. Add new fixtures when production corrections happen. If a human fixes a table value, turn that exact failure into a regression check.

Score capabilities separately

Do not collapse everything into one “parser quality” number. Agents fail in specific ways, so score specific capabilities.

Check type	What to test	Why it matters
Table cell	exact value, row, column, unit	prevents wrong financial extraction
Chart fact	cited trend or value	prevents unsupported visual summaries
Layout order	heading, paragraph, caption sequence	prevents mixed context
Evidence	page and bbox available	enables human verification
Schema	output validates before write	protects downstream systems

This is also where the product loop should connect back to PDF to JSON and schema extraction. If a parser returns impressive text but cannot satisfy the schema, the agent still cannot use it safely.

Separate scoring also makes tradeoffs visible. One parser may produce beautiful markdown but weak cell provenance. Another may preserve cells but miss captions. A third may be too expensive for the whole document but perfect for disputed pages. Agent systems need that matrix, not a single vendor score.

Compare by task, not by model name

The question is not whether a model is “best.” The question is which parser configuration wins for a task under your constraints.

For example:

• Use text-layer extraction for clean public filings when the task is keyword search.
• Use a VLM parser for scanned invoices with tables.
• Use bbox citation generation for workflows that require human signoff.
• Use a cheaper fallback for pages that do not affect the answer.

That is how document parsing becomes an engineering system instead of a leaderboard argument.

Task-based comparison also protects you from overfitting to demos. A parser that handles a glossy annual report may fail on a faxed medical form. A parser that handles invoices may miss chart-heavy investor decks. The only stable decision is to test the document classes your agents actually touch.

Implementation shape for agent tools

Start with a minimal parser contract

An agent tool should expose a simple contract: upload or resolve a PDF, inspect available facets, query structured nodes, and fetch visual evidence when needed.

curl -X POST https://api.okrapdf.com/v1/files \
  -H "Authorization: Bearer $OKRA_API_KEY" \
  -F "file=@filing.pdf"

curl -X POST https://api.okrapdf.com/v1/parse \
  -H "Authorization: Bearer $OKRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "file": { "id": "file_..." },
    "outputs": { "markdown": true, "json": true, "tables": true }
  }'

The exact endpoint shape will differ by platform, but the contract should not. Agents need stable document identity, available facets, queryable structure, and evidence.

For MCP-style tools, this contract can be even smaller. The agent should first discover what documents and facets exist, then query structured nodes, then request visual evidence only when needed. That keeps the model context compact and lets the tool own the expensive document work.

Query nodes before asking the model

Once the parser emits nodes, the agent should query them directly before spending model tokens.

select page_number, type, value
from nodes
where document_id = :document_id
  and type in ('heading', 'paragraph', 'table', 'cell')
  and value like '%revenue%'
order by page_number, sort_order;

That makes the model a reasoning layer over retrieved evidence, not a blind OCR engine. The same idea is behind our argument to stop writing regex for PDF extraction: use document structure first, then apply code and models where they fit.

This also makes debugging practical. If the agent returns a wrong answer, you can inspect the SQL result, the node IDs, the parser payload, and the page image. You are not stuck asking whether the model “understood” a PDF that never had a stable intermediate representation.

Route uncertainty into review

Agents should not pretend every parse is complete. A production workflow should route weak evidence into a review path.

function shouldAskForReview(field) {
  if (!field.bbox) return true;
  if (field.confidence != null && field.confidence < 0.82) return true;
  if (field.sourceText && field.sourceText.length > 400) return true;
  return false;
}

The review surface can be simple: show the page image, highlight the bbox, show the extracted value, and let a human accept or correct it.

The key is that review should feed the parser loop, not sit outside it. A correction should update the saved field, attach to the evidence record, and become a candidate eval case. That is how an agent workflow gets better without trusting every future page blindly.

Where okraPDF fits

Treat documents as agent tools

okraPDF is moving toward the same conclusion: a PDF should not be a blob handed to a model. It should be a tool surface with facets, queries, citations, and hosted views.

That is why the agent workflow matters. A coding agent using a PDF skill, like the pattern in okraPDF with Claude Skills, should call deterministic document tools first and then reason over the returned evidence.

The product implication is simple: the document tool should be useful before chat enters the loop. Upload, inspect, query, cite, and display should all work as normal API operations. Chat is valuable after the evidence exists.

Keep evaluation close to the parser

The next useful product layer is not another generic chat box. It is evaluation close to the parser: page fixtures, capability checks, saved result sets, and deterministic scoring inspired by ParseBench.

{
  "eval_suite": "bank-statement-agent-v1",
  "document_class": "bank_statement",
  "checks": ["table_cell", "running_balance", "date_normalization"],
  "required_evidence": ["page", "bbox", "source_text"]
}

That is how teams decide whether a parser is good enough for an agent to use without pretending a benchmark can replace their own documents.

The best version is a living eval suite per document class. Bank statements get balance checks. Contracts get clause and date checks. Research papers get citation and table checks. Financial filings get chart, table, and footnote checks. The parser roadmap then follows measured failures instead of vibes.

Ship the failure loop

The production loop is straightforward:

1. Ingest the PDF once.
2. Generate parser facets.
3. Run task-specific checks.
4. Store evidence with each extracted field.
5. Let agents query nodes and artifacts.
6. Route weak evidence into review.
7. Feed corrections back into the eval suite.

ParseBench and dots.mocr are useful because they make that loop more concrete. The future is not just better OCR. It is document parsing that gives agents the evidence, structure, and control surface they need to do real work.

That is the standard worth building toward: not “the model read the PDF”, but “the agent can inspect the document, verify the source, and safely act on the result.”