Document Parsing for AI Agents: Docling, LlamaParse, LiteParse, and Ollama, Step by Step

The wrong way to pick a document parser for an AI agent is to read a leaderboard screenshot and install the winner. The right way takes about an hour: run the candidate parsers on pages with known correct answers and read the diffs yourself.

ParseBench (LlamaIndex, 2026) makes that hour easy, because it ships both halves: ~2,000 real enterprise PDF pages and human-verified expected output for each one, organized into the five capabilities that break agent workflows — tables, charts, content faithfulness, semantic formatting, and visual grounding. The dataset and eval code are open.

This post walks through four of the most used parsing stacks in the industry — Docling, LlamaParse, LiteParse, and a local VLM on Ollama — step by step, all against the same ParseBench page, so every claim here is something you can reproduce with the commands shown.

Disclosure: okraPDF is a document-parsing platform. We don’t sell any of the parsers below — routing across parsing engines is a core part of our product, so we run these benchmarks for our own engineering decisions and publish what we find. Every output excerpt in this post came from running the tools ourselves — Docling and LiteParse on a MacBook, Qwen3-VL on an RTX 2070 SUPER desktop — except LlamaParse, which is hosted; for it we show the exact code and the benchmark’s published scores.

Table of Contents

• The bench: one page, one verified answer
• Step 1: Docling
• Step 2: LlamaParse
• Step 3: LiteParse
• Step 4: Ollama + Qwen3-VL
• Same cell, four parsers
• The scoreboard
• Score any parser yourself
• Choosing by task
• Connect the winner to your agent over MCP
• Connect parsers to agent CLIs
• Where okraPDF fits
• Frequently asked questions

The bench: one page, one verified answer

Grab the benchmark repo — it includes a small --test split (a few documents per category) that is perfect for hands-on comparison:

git clone https://github.com/run-llama/ParseBench
cd ParseBench

Our running example is data/test/docs/table/222876fb_page22.pdf: page 22 of a Linear Technology LTC2228 ADC datasheet. It is a great agent-parsing stress test in miniature — two dense text columns, a footer, ligatures, and one table where subscripts carry meaning: the MODE pin levels 1/3V<sub>DD</sub> and 2/3V<sub>DD</sub> are voltages, not the strings “1/3V DD” or “13VDD”.

The human-verified answer lives next to it in data/test/table.jsonl (lightly reformatted here):

<table>
  <tr><th>MODE PIN</th><th>OUTPUT FORMAT</th><th>CLOCK DUTY CYCLE STABILIZER</th></tr>
  <tr><td>0</td><td>Offset Binary</td><td>Off</td></tr>
  <tr><td>1/3V<sub>DD</sub></td><td>Offset Binary</td><td>On</td></tr>
  <tr><td>2/3V<sub>DD</sub></td><td>2's Complement</td><td>On</td></tr>
  <tr><td>V<sub>DD</sub></td><td>2's Complement</td><td>Off</td></tr>
</table>

That one table exercises three of ParseBench’s five dimensions at once: table structure, semantic formatting (the subscripts), and content faithfulness (no invented or dropped rows). Keep it in view as we run each parser.

Step 1: Docling

Docling is IBM’s open-source pipeline: layout detection plus TableFormer table-structure models, CPU-native, no API key, Apache-licensed. Install and run:

pip install docling
docling 222876fb_page22.pdf --to md --output out/

Or in Python:

from docling.document_converter import DocumentConverter

result = DocumentConverter().convert("222876fb_page22.pdf")
print(result.document.export_to_markdown())

The first invocation downloads the model weights (a couple of GB); after that it runs fully offline. Warm, the CLI took about 13 seconds for this one page on a laptop CPU — most of that is per-invocation model loading, which a long-running Python process amortizes away.

Here is what Docling 2.75 actually produced for the table region on our page:

Table 2. MODE Pin Function

| MODE PIN   | OUTPUT FORMAT   | CLOCK DUTY CYCLE STABILIZER   |
|------------|-----------------|-------------------------------|
| 0          | Offset Binary   | Off                           |
| 1/3V DD    | Offset Binary   | On                            |
| 2/3V DD    | 2's Complement  | On                            |
| V DD       | 2's Complement  | Off                           |

Read it against the ground truth:

• Structure: correct. Three columns, four data rows, headers intact. An agent doing cell lookups gets the right grid.
• Formatting: lost. 1/3V<sub>DD</sub> became 1/3V DD — the subscript survives only as a stray space. This is Docling’s known blind spot; it scores 1.03/100 on ParseBench’s semantic-formatting dimension while scoring a solid 66.41 on tables.
• Character-level noise. Elsewhere on the page it emitted L TC2228 for LTC2228 and split the fl ligature in “Overflow” into Overfl ow. Harmless for a human reader, hostile to an agent doing string matching.

Docling also rendered the page’s logo graphic inline as a base64 data: URI — nice for fidelity, something to strip before feeding an LLM context window.

Step 2: LlamaParse

LlamaParse is the hosted parser from LlamaIndex, and the current ParseBench leaderboard leader. It is also made by the same company that maintains the benchmark — the eval code is open, so you can re-run their numbers, but keep the relationship in mind.

Get a key at cloud.llamaindex.ai, then:

pip install llama-cloud
export LLAMA_CLOUD_API_KEY=llx-...

import os
from llama_cloud import LlamaCloud

client = LlamaCloud(api_key=os.environ["LLAMA_CLOUD_API_KEY"])

job = client.parsing.create(
    upload_file="222876fb_page22.pdf",
    tier="cost_effective",   # or "agentic" / "agentic_plus"
    version="latest",
)
client.parsing.wait_for_completion(job.id, timeout=600)

result = client.parsing.get(job.id, expand=["items", "text", "metadata"])

This is the exact call shape ParseBench’s own harness uses (see providers/parse/llamaparse.py in the repo). Two details matter for agent work:

• tier is the cost/quality dial. cost_effective runs at 0.375¢ per page and scores 76.77 overall; agentic runs an agentic parse-verify-reextract loop at 1.25¢ per page and scores 84.88 — the top overall score, and the top score on tables (90.74) and semantic formatting (85.24). On our test table, that formatting score is the difference between getting V<sub>DD</sub> back with the subscript intact and getting a flattened string.
• expand=["items", ...] is the agent-facing output. Instead of one markdown blob, items returns typed page elements — markdown, text, and bounding boxes per element — which is what you need to cite a table cell back to a page region.

The tradeoffs are the obvious ones: documents leave your machine, you pay per page, and throughput is a queue rather than a local process.

Step 3: LiteParse

LiteParse (@llamaindex/liteparse) is LlamaIndex’s local parser: a native PDFium-based engine with prebuilt binaries for macOS/Linux/Windows, no network, no key. It is the “deterministic text-layer” end of the spectrum:

npm install @llamaindex/liteparse

import { readFileSync } from "node:fs";
import { LiteParse } from "@llamaindex/liteparse";

const pdf = readFileSync("222876fb_page22.pdf");
const parser = new LiteParse({ outputFormat: "markdown", ocrEnabled: false });
const result = await parser.parse(pdf);
console.log(result.text);

On our page it finished in 727 ms — three orders of magnitude faster than a VLM — and produced this for the table:

**Table 2. MODE Pin Function**

| MODE PIN | OUTPUT FORMAT | CLOCK DUTY CYCLE STABILIZER |
|---|---|---|
| 0 | Offset Binary | Off |
| 1/3VDD | Offset Binary | On |
| 2/3VDD | 2's Complement | On |
| VDD | 2's Complement | Off |

Structure: perfect. Subscripts: flattened harder than Docling — 1/3V<sub>DD</sub> became 1/3VDD with no separator at all, and the page number 22 leaked into the reading flow as a paragraph. Same ligature split in “Overflow”.

Where LiteParse earns its place is evidence. Flip on extractBlocks and every table cell comes back with a real bounding box, from the PDF’s own geometry rather than a model’s guess:

const blocks = await new LiteParse({ extractBlocks: true }).parse(pdf);
console.log(blocks.pages[0].blocks.find((b) => b.kind === "table"));

{
  "kind": "table",
  "header": [
    { "text": "MODE PIN",
      "bbox": { "x": 45.0, "y": 390.8, "width": 83.9, "height": 24.5 } },
    { "text": "OUTPUT FORMAT",
      "bbox": { "x": 128.9, "y": 390.8, "width": 83.9, "height": 24.5 } }
  ],
  "rows": [ "..." ]
}

That is a page-region citation for free, in under a second, with zero tokens spent. The catch is the flip side of determinism: with OCR off it reads only the text layer, so scanned pages return nothing, and it makes no attempt at charts. Its ParseBench overall of 32.8 (tables 40.3, charts 3.4) reflects exactly that — the benchmark’s insurance and government documents are heavy on scans and charts. On born-digital pages like ours, it punches far above that number.

Step 4: Ollama + Qwen3-VL

The fully-local VLM route: Ollama serving Qwen3-VL 8B, the model the ParseBench paper found to be the best local all-rounder — 61.97 overall at BF16, with no catastrophic dimension. We ran this step on a desktop with an RTX 2070 SUPER (8 GB VRAM); higher-scoring open-weight parsers exist on the leaderboard (MinerU 2.5 Pro, Infinity-Parser2), but they want a vLLM server, not a one-line pull.

ollama pull qwen3-vl:8b   # 6.1 GB, Q4_K_M quant

VLMs eat images, not PDFs, so render the page first:

import pypdfium2 as pdfium

page = pdfium.PdfDocument("222876fb_page22.pdf")[0]
page.render(scale=150 / 72).to_pil().save("page22.png")  # ~150 DPI

Then call Ollama’s chat API with the image attached:

import base64, json, urllib.request

img = base64.b64encode(open("page22.png", "rb").read()).decode()
body = json.dumps({
    "model": "qwen3-vl:8b",
    "stream": False,
    "think": False,
    "options": {"temperature": 0, "num_predict": 8192, "num_ctx": 12288},
    "messages": [{
        "role": "user",
        "content": "Transcribe this document page into markdown. Keep headings, "
                   "paragraphs, and the table (as a markdown table). "
                   "Use <sub> tags for subscripts like VDD. /no_think",
        "images": [img],
    }],
})
req = urllib.request.Request("http://localhost:11434/api/chat", data=body.encode(),
                             headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["message"]["content"])

Three settings in that request are not optional, and we lost an hour to them before writing this:

• "num_ctx": 12288 — Ollama’s default context window is 4,096 tokens, and the page render alone consumes 2,000–3,700 of them depending on DPI. Our first runs all died at ~390 output tokens with done_reason: "length", mid-table. If your VLM transcripts keep truncating, this is why. Size the window to your memory — jumping straight to 16k plus a page image took the Ollama desktop app down entirely on one of our machines; 12,288 with a 150-DPI render is the configuration that held.
• Reasoning happens whether you ask or not. Qwen3-VL is a thinking model, and neither "think": false nor the /no_think prompt tag actually stopped it in our runs — it privately reasons through the whole page (“the table has columns: MODE PIN, OUTPUT FORMAT…”) before emitting a word of transcript. On this page that was ~5,000 hidden tokens before ~1,200 tokens of output.
• "num_predict": 8192 follows from that: the budget has to cover the reasoning and the transcript. At 4,096 the model burned the entire budget thinking and returned an empty content — which looks exactly like a broken install until you inspect the response’s thinking field.

Configured that way, here is what Qwen3-VL 8B produced for the table — the actual output, unedited:

**Table 2. MODE Pin Function**

| MODE PIN | OUTPUT FORMAT | CLOCK DUTY CYCLE STABILIZER |
|----------|---------------|-----------------------------|
| 0        | Offset Binary | Off                         |
| 1/3V<sub>DD</sub> | Offset Binary | On                          |
| 2/3V<sub>DD</sub> | 2's Complement | On                          |
| V<sub>DD</sub> | 2's Complement | Off                         |

That is the only local output in this post that matches the ground truth all the way down to the subscripts. No ligature splits either — “Overflow” survived intact. The VLM’s failure mode is different in kind: elsewhere on the page it silently rewrote “overranged” as “over-ranged”. A deterministic parser garbles text visibly; a VLM edits it invisibly, which is worse for an agent that quotes its source.

Two more caveats from this run. First, the local number is not the leaderboard number: Ollama’s default build is a 4-bit quant, and the 61.97 score was measured at BF16 on server GPUs — treat local quality as a notch below. Second, latency: the full page took about 11 minutes on an RTX 2070 SUPER (the 8 GB card spills a fifth of the weights to CPU, ~15 tokens/s, and five of every six generated tokens were hidden reasoning). For a 300-page filing, that is the difference between an interactive tool and a multi-day batch job.

Same cell, four parsers

The ground-truth cell is 1/3V<sub>DD</sub>. Here is what each stack did with it on our machines:

Parser	Cell output	Table structure	Time on this page
Ground truth	1/3V<sub>DD</sub>	—	—
Docling 2.75	1/3V DD	correct	~13 s (CLI, warm, laptop CPU)
LlamaParse	run it with your key — tiers above	leaderboard tables: 90.74 (agentic)	hosted queue
LiteParse	1/3VDD	correct	0.7 s
Qwen3-VL 8B (Ollama, Q4)	1/3V<sub>DD</sub> ✓	correct	~11 min (RTX 2070S)

One page is an anecdote, not a benchmark — that’s what the full harness below is for. But the anecdote maps cleanly onto the scoreboard: everyone got the grid right, and the meaning of the cell survived in exact proportion to what you paid. The deterministic parsers returned in seconds and quietly turned a voltage subscript into a different string. The VLM kept the subscript perfectly and took eleven minutes — while rewording a term elsewhere on the page. Speed, semantics, faithfulness: on this page no local stack gives you all three, which is the strongest argument for routing per page instead of picking one winner.

The scoreboard

Full-benchmark scores for the stacks in this post, from the ParseBench leaderboard (higher is better; costs are per page):

Parser	Overall	Tables	Charts	Faithfulness	Formatting	Grounding	Cost
LlamaParse Agentic	84.88	90.74	78.11	89.68	85.24	80.62	1.25¢
LlamaParse Cost Effective	76.77	81.42	70.15	90.92	68.78	72.59	0.375¢
Qwen3-VL 8B (BF16)	61.97	74.61	28.18	87.63	64.23	55.18	local
Docling	50.65	66.41	52.76	66.93	1.03	66.11	local
LiteParse (no OCR)	32.8	40.3	3.4	68.6	44.6	10.7	local

How to read this like an engineer rather than a shopper:

• The overall column hides the plot. Docling beats Qwen3-VL on charts and grounding while losing by 11 points overall. LiteParse looks last and is still the only local option here that returns geometric bounding boxes in under a second.
• Every dimension is a different failure mode for your agent. Tables → wrong financial values. Formatting → struck-through prices read as current. Grounding → no way for a human to verify a citation. Match the column to what your agent actually does.
• The hosted gap is real but priced. LlamaParse Agentic’s 23-point lead over the best local option costs 1.25¢ per page and a round trip of your documents to someone else’s cloud. Whether that trade is good depends on your volume and your compliance posture, not on the leaderboard.

We covered the local-only end of this table in more depth — including Dots OCR, MinerU, and what fits in 16 GB of RAM — in our local PDF parsing benchmark post.

Score any parser yourself

Everything above is one page. The same repo scores any parser on all ~2,000 pages, with the same five-dimension report the leaderboard uses:

cd ParseBench
uv sync --extra runners

# smoke test on the small split (a few docs per category)
uv run parse-bench run llamaparse_cost_effective --test

# other pipelines from this post
uv run parse-bench run docling_serve --test      # needs a docling-serve endpoint
uv run parse-bench run liteparse_markdown --test # local, no key

# interactive per-page report in your browser
uv run parse-bench serve llamaparse_cost_effective

Each pipeline reads its key or endpoint from .env (LLAMA_CLOUD_API_KEY, DOCLING_SERVE_ENDPOINT_URL, and so on — docs/pipelines.md in the repo lists all 90+). The serve dashboard is the part worth your time: it shows the parsed output next to the expected answer for every page, which is how you find your document class’s failure modes instead of the average one.

The pattern to steal even if you never run ParseBench itself: keep a small set of your own pages with hand-verified answers, and run every parser change against it. A ten-page fixture set with five hard checks beats a thousand-page corpus nobody reads. When production corrects a value, that page becomes a new fixture. That loop — parse, check against known answers, route failures to review — is the whole game; the benchmark just industrializes it.

Choosing by task

• Born-digital PDFs, high volume, latency matters → LiteParse. Sub-second, deterministic, real bboxes. Add an OCR server (ocrServerUrl) before it meets scans.
• Chart-heavy or scanned enterprise docs, no cloud allowed → Docling for charts/grounding, Qwen3-VL on Ollama for prose and formatting-sensitive pages; route by page type, and spot-check the VLM’s text against the text layer — it edits words silently.
• Formatting carries legal or financial meaning (redlines, struck prices, footnote markers) → the local stacks all lose it; use LlamaParse Agentic or route those pages to a human.
• Best raw quality, cost and cloud acceptable → LlamaParse Agentic, and pin version so results stay reproducible.
• Building an agent tool, any parser → keep the evidence. Whatever parses the page, store page number, bbox, and source text with every extracted field, or your agent can’t show its work when it matters.

Connect the winner to your agent over MCP

Everything above produces structure; the remaining question is how an agent consumes it. Model Context Protocol is the wire format that hands a document surface to an agent without writing a client per agent — instead of teaching your agent an HTTP API, you point it at a server and it discovers the document tools by name.

okraPDF runs one at https://okrapdf.com/mcp over Streamable HTTP, so any MCP-capable client connects with a URL and no local install:

{
  "mcpServers": {
    "okrapdf": {
      "type": "http",
      "url": "https://okrapdf.com/mcp"
    }
  }
}

The tool names map onto the stages this post has been walking manually:

Stage	Tool	What the agent gets back
Resolve a document	resolve_pdf_url, upload_document	Stable document identity to reference in later calls
Read it as structure	view_structured, view_document	Semantic nodes — headings, paragraphs, tables, cells — not a flat transcript
Make it readable	view_html	A streaming, screen-reader-friendly HTML twin at a viewer_url
Check a claim	verify_source, verify_block	Page and bbox evidence for a specific extracted value
Route the doubtful cases	review_extraction	A human-review hand-off for weak evidence

One honest caveat: MCP gives your agent a stable interface to documents, not immunity from the parser failures shown above. A tool call that returns 1/3VDD instead of 1/3V<sub>DD</sub> is still wrong — but now the wrongness is inspectable: replay the call, look at the nodes, and pin the failure to a stage instead of to “the model misread the PDF.”

Connect parsers to agent CLIs: Claude Code, Codex, Gemini CLI, Cursor

The MCP section above is one config block away from every major agent CLI. The server is the same in each case — https://okrapdf.com/mcp, no API key — so the only thing that changes is where the config lives.

Claude Code

One command registers the server:

claude mcp add --transport http okrapdf https://okrapdf.com/mcp

Then steer the agent in CLAUDE.md the way you would document any tool contract:

## PDF handling
- Resolve or upload documents with the okrapdf MCP tools before reasoning about them.
- Query structured nodes first; do not paste raw PDF text into context.
- Any extracted number must carry page + bbox evidence before it is written anywhere.

That last rule is only enforceable because the parser returns page and bbox with every field — the same evidence argument as the LiteParse extractBlocks and LlamaParse expand=["items"] examples above. Claude Desktop users can add the same URL as a custom connector under Settings → Connectors (walkthrough), and teams that want repeatable document workflows can package them as Claude Skills.

OpenAI Codex CLI

Codex CLI reads MCP servers from ~/.codex/config.toml. The portable pattern is the mcp-remote bridge:

[mcp_servers.okrapdf]
command = "npx"
args = ["-y", "mcp-remote", "https://okrapdf.com/mcp"]

Newer Codex builds can also connect to streamable HTTP servers directly — check codex mcp --help for what your version supports.

Gemini CLI

Gemini CLI configures MCP in ~/.gemini/settings.json (or a project-level .gemini/settings.json), with httpUrl for streamable HTTP servers:

{
  "mcpServers": {
    "okrapdf": { "httpUrl": "https://okrapdf.com/mcp" }
  }
}

Cursor

Cursor reads .cursor/mcp.json in the project (or ~/.cursor/mcp.json globally):

{
  "mcpServers": {
    "okrapdf": { "url": "https://okrapdf.com/mcp" }
  }
}

Any other agent with shell access

Aider, OpenHands, a LangGraph loop, or a plain cron job can skip MCP entirely and run the parsers from this post as libraries — LiteParse from Node, Docling from Python, Qwen3-VL against a local Ollama daemon — exactly as shown in the steps above. That is the right fit when the “agent” is a pipeline rather than an interactive CLI session.

The pattern worth noticing: the CLIs differ only in config syntax. The document surface — stable IDs, structured nodes, bbox evidence — is identical across all of them. That is what makes parser quality a platform decision instead of a per-agent hack.

Where okraPDF fits

okraPDF’s position in this landscape is the router, not another engine: we treat parsers as pluggable vendors with per-dimension capability profiles and send each page to the engine that wins for it — the same logic this post walks through manually. You can watch that routing happen in the playground, or get the end product — structured JSON with page-level grounding from whatever engine mix the document needed — from the PDF to JSON tool.

If you’re building the agent side, the argument for structure-first parsing over prompt-and-pray is the same one we made in stop writing regex for PDF extraction, and the pattern for wiring parsed documents into a coding agent is in okraPDF with Claude Skills.

Frequently asked questions

What is document parsing for AI agents?

Document parsing for AI agents converts a PDF into structured, verifiable artifacts — page images, text spans, layout regions, tables, chart data, and bounding-box citations — that an agent can query and act on. It differs from plain OCR, which returns text without the structure or evidence an agent needs to calculate, cite, or write data safely into other systems.

Which of the four parsers should I start with?

LiteParse if your PDFs are born-digital and latency matters, Docling if they’re scanned or chart-heavy, Ollama + Qwen3-VL if you want one local model with no catastrophic weakness, LlamaParse if quality beats data-locality. The honest answer is measured, not chosen: run the --test split above on your own document class first.

What is ParseBench?

ParseBench is a document-parsing benchmark from LlamaIndex that scores parsers on the capabilities that matter for real enterprise PDFs — tables, charts, semantic formatting, visual grounding, and faithfulness — instead of a single OCR-accuracy number. It ships ~2,000 pages with human-verified expected output, which is what every example in this post is checked against.

Is LlamaParse’s leaderboard lead trustworthy, given LlamaIndex runs the benchmark?

The dataset, ground truth, and scoring code are public, and the inclusion rules are documented in the repo. We reproduced MinerU 2.5’s score ourselves for the local-parser post and it held. Skepticism is still healthy — which is why this post shows you how to re-run everything.

Why does my local Qwen3-VL output differ from the leaderboard score?

Quantization (Ollama defaults to 4-bit; the 61.97 score is BF16 on server GPUs), rendering DPI, context window, and prompt all move the number. Treat leaderboard VLM scores as upper bounds for local quants.

How do I give Claude Code or another agent CLI access to a PDF parser?

Register a document-parsing MCP server with the CLI. For Claude Code, claude mcp add --transport http okrapdf https://okrapdf.com/mcp — no API key needed. Codex CLI, Gemini CLI, and Cursor take the equivalent entry in their MCP config files; see the per-CLI examples above.

Do agents still need RAG if documents are parsed?

Retrieval still helps for finding the right document and section, but parsing changes what gets retrieved: typed nodes with page evidence instead of text chunks. Many agent tasks — table lookups, field extraction, verification — become deterministic queries over parsed nodes and never touch a similarity search at all.

What happened to the okraPDF REST API examples that used to be in this post?

Those api.okrapdf.com endpoints were retired. The current supported surfaces are the MCP server at okrapdf.com/mcp, the tools (including PDF to JSON), and the playground; this post now focuses on the industry parsers you can run yourself.