Reading a document is the simplest task to describe and the easiest to get wrong.

CER / WER are edit distance over characters or words, normalised by reference length. 1% CER ≈ one wrong character per hundred. Lower is better. They reward raw transcription accuracy but say nothing about structure or order.

OmniDocBench reports a text edit distance over the whole page (lower is better), so reading-order mistakes are penalised, not just glyph errors. OCRBench uses a 0–1000 composite over short tasks; OCRBench v2 adds private EN/ZH splits to resist contamination.

1 · Detect the text layer (PyMuPDF)

import fitz  # PyMuPDF

def has_text_layer(page, min_chars: int = 12) -> bool:
    """A born-digital page returns real characters; a scan returns ~nothing."""
    return len(page.get_text("text").strip()) >= min_chars

def read_pdf(path: str) -> list[str]:
    """Parse where there is a text layer, OCR only where there isn't."""
    doc = fitz.open(path)
    out = []
    for page in doc:
        if has_text_layer(page):
            # Perfect, lossless text — never OCR this.
            out.append(page.get_text("text"))
        else:
            # Scanned page → rasterise and hand to an OCR model.
            pix = page.get_pixmap(dpi=200)
            out.append(ocr_image(pix.tobytes("png")))
    return out

2 · OCR the scanned pages to Markdown

import replicate, base64

def ocr_image(png_bytes: bytes) -> str:
    """Scanned page → Markdown. Reading order + structure preserved."""
    b64 = base64.b64encode(png_bytes).decode()
    output = replicate.run(
        "rednote-hilab/dots.ocr",
        input={
            "image": f"data:image/png;base64,{b64}",
            "prompt": "Convert this page to Markdown. Preserve reading order, "
                      "headings, lists and tables. Do not invent text.",
            "temperature": 0.0,
        },
    )
    return "".join(output)

Layout-driven errors

• ›Reading order. Multi-column pages get interleaved into nonsense unless the model recovers column order first — a layout problem upstream of reading.
• ›Headers, footers, marginalia. Running heads and page numbers get spliced into body text.
• ›Tables in prose. A table read left-to-right as a sentence destroys the data — see the tables task.

Transcription errors

• −VLM hallucination. Generative models can “complete” faded text plausibly but wrongly. Use temperature 0 and an explicit “do not invent” instruction.
• −Low resolution. Below ~150 DPI, glyph confusions spike (rn→m, cl→d, 0→O).
• −Diacritics & scripts. Polish, Vietnamese, Arabic and CJK still trail Latin English; check the ZH split, not just EN.
• −Rotation & skew. Phone photos at an angle need deskewing before OCR.