Handwriting OCR benchmark: okraPDF reads IAM at 1.29% CER

“How good is handwriting OCR, really?” is a question that usually gets answered with a vendor screenshot of one clean note. So we measured it properly: four systems, the same handwritten lines, the same scoring.

The short version: okraPDF reads handwriting at 1.29% character error rate on the IAM test set — the standard offline-handwriting benchmark. That puts it in the “excellent” band, ahead of the open specialist model and roughly 38× lower error than classic OCR.

The result

We sampled 120 lines from the IAM-line test split (the canonical handwriting benchmark, MIT-licensed, hosted on Hugging Face), sent the identical image to each system, and scored them the same way.

System	Class	CER	WER	Lines read exactly
okraPDF (/v1 production)	document parser	1.29%	6.18%	67.5%
Gemini 3 Flash	general vision model	1.42%	6.65%	64.2%
TrOCR-large-handwritten	specialist HTR (open weights)	3.68%	11.24%	45.0%
Tesseract 5	classic OCR	49.58%	91.1%	0%

CER (character error rate) is the share of characters you’d have to insert, delete, or substitute to fix the output. Lower is better. The published quality bands (arXiv:2503.15195) read: under 2.5% is excellent, under 5% very good, 5–10% good, over 10% unusable.

Two things stand out. First, handwriting destroys traditional OCR — Tesseract, which is fine on printed text, lands at ~50% CER and gets zero lines exactly right. If your stack is Tesseract and your documents have handwriting, the handwritten fields are noise. Second, a document parser and a frontier vision model are now neck-and-neck at the top, both comfortably inside the “excellent” band.

What it looks like

Here’s one line every system saw:

• okraPDF: CHRIS CHARLES, 39, who lives in Stockton-on-Tees, is an accountant. ✓
• Gemini 3: ... Stocuton-on-Tees ...
• TrOCR: ... Stoculon-on-Tees ...
• Tesseract: CHRIS CHARLES, 33, who lives in Stocuton-on ~ Tees isan accountant.

okraPDF was the only system to read both the town and the number correctly. That is the difference that matters downstream — a misread proper noun or a 39→33 slip is the kind of error that quietly breaks an extraction pipeline or an audit trail.

The methodology (and one fairness trap)

The benchmark is only as good as its scoring, and IAM has a trap worth naming.

IAM’s ground truth is tokenized: punctuation is spaced out and contractions are split, so the reference for that line is literally CHRIS CHARLES , 39 , ... is an accountant . Specialist models trained on IAM reproduce that convention. A general system — okraPDF, a VLM — outputs natural text (it's, not it 's). If you score raw, you penalize the general systems for formatting they were never asked to imitate, not for misreading anything.

So we normalize: detokenize the prediction and the ground truth identically before scoring. The number above is “did it read the handwriting,” not “did it guess IAM’s spacing convention.”

How much does that matter? On raw (un-normalized) CER, the ranking partly inverts — TrOCR edges ahead at 3.76% because it matches IAM’s tokenization, while okraPDF and Gemini look worse purely from punctuation spacing. That inversion is the artifact normalization removes. Any benchmark that quotes a single IAM CER without saying how it normalized is hiding this.

The rest of the setup, plainly:

• Dataset: 120 lines from Teklia/IAM-line test split — unconstrained English cursive and print from 600+ writers.
• Metric: CER and WER, micro-averaged Levenshtein edit distance against the ground truth.
• Same input, same harness: the exact same line image went to every system; TrOCR-large ran locally, the rest via their production APIs.
• Directional: N=120, single run per line. Not a leaderboard submission — a same-harness read on where these systems actually stand.

Where this sits in the literature

The published state of the art on IAM (line-level), for context — different splits and preprocessing, so treat it as a map, not a head-to-head:

System	What it is	IAM CER	Source
HTR-JAND	best published specialist	1.23%	arXiv:2412.18524
GPT-4o-mini (zero-shot)	best published VLM	1.71%	arXiv:2503.15195
TrOCR-large	open-weight baseline	2.89%	arXiv:2109.10282

A skilled human transcriber runs roughly 1–4% CER depending on the source (the often-quoted “2.5% human error” is from MNIST digits, not handwritten prose — don’t reuse it for text). So modern handwriting OCR on clean modern hands is now inside the human transcription band.

What about the commercial OCR vendors? None of Google Document AI, AWS Textract, or Azure publishes a CER on IAM. The only numbers that exist are third-party single-page tests (handwriting WER roughly 9–23%, n=1, not IAM) — useful as a smell test, not a benchmark. That absence is itself the point: “handwriting supported” on a feature page is not an accuracy claim.

Caveats

This is modern English handwriting. The honest edge of the envelope:

• Historical and degraded hands are still hard. Frontier VLMs that score ~1.7% on IAM collapse to 70–98% CER on 18th-century German archival material; specialist models trained on the period still win there. Benchmark on documents that look like yours.
• It’s a directional sample, not the full 2,915-line test set. The ranking is stable and the gaps are large, but treat the decimals as approximate.

Read handwriting in your pipeline

If you have scanned notes, filled forms, ledgers, or letters going through a PDF workflow, the same /v1 parser that produced these numbers reads them — page reading order, tables, and bbox citations included.

• Try it on the live numbers and sample lines: Handwriting OCR tool
• Want the architecture and trade-offs behind HTR? Read the developer’s guide to handwritten text recognition.

One call:

okra upload handwritten-note.pdf   # → doc-…
okra read doc-…                    # → the transcription