The ATS Parse Benchmark

Contact details in the header or footer simply vanish

Across the 5 corpus documents that carry their email address and phone number in the Word header/footer region — a layout most template marketplaces ship — not one of those 10 contact strings survived text extraction. The body text arrived intact; the person it belongs to became unreachable.

The parsed record makes the failure concrete. On the header-contact document, the extractor found no email address at all, and then did something worse than nothing: the phone-number field filled itself with "2013 - 2017" — the education date range, misread as a phone number. A recruiter exporting contact details from that record would dial a degree.

Anything drawn instead of typed does not exist

Two corpus documents carry part of their skills list only inside an embedded image — a skills chart, the kind design-led templates render as bars or badges. Every skill that existed only as pixels (4 of 4 across the two documents) was absent from the extracted text. Text extraction does not OCR images; it discards them.

On the graphics document, the structure scan did detect the embedded image, and the record's skills field was left holding only the section heading with nothing under it — the parser knows a Skills section was announced, and receives no skills.

Unusual section headings lose nothing — and map almost nothing

The creative-headings document is the corpus's most instructive failure, because nothing is lost: every canary string survived extraction. But with "My Journey" in place of Work Experience and "What I Bring" in place of Skills, the field-mapper recognised no section to file the content under — 19 of its 24 lines (79.2%) landed in no database field, and only 4 of 8 standard fields filled, against 8 of 8 for the identical content under conventional headings.

This is the quiet failure mode candidates never see: the resume reads beautifully to a human, extracts perfectly, and produces a database record that is mostly empty. Searchable is not the same as extracted.

Tables and columns scramble reading order more than they delete text

On this parsing path, table-based layouts mostly kept their text: the constructed tables document and the constructed two-column document each lost 0 canaries. What broke was order and mapping. The two-column document's sections were read sidebar-first — SKILLS, EDUCATION, CERTIFICATIONS, SUMMARY, WORK EXPERIENCE — so the skills and education arrive before the summary and experience the layout puts front and centre.

The seed documents, built the way real-world templates actually are, fared worse: the two-column CV filled 5 of 8 record fields with its summary field empty and its contact details (header-borne) gone; the table-based resume filled 6 of 8, with its structure scan flagging 3 separate tables and its free health score landing at 44 — the corpus's lowest.

Honest caveat: Different extraction engines linearise tables differently — some scramble cell order far more aggressively than this path does. Our figures describe our extractor; the hazard flags exist precisely because behaviour varies across parsers.

Non-standard date formats halve the recoverable timeline

The audit engine's chronology parser recovered every date range on the control document (4 of 4) and only 2 of 4 on the same document with its role dates rewritten into three non-standard formats. Two of the three employment ranges — and with them tenure, gaps and role order — became uncomputable.

The subtler finding: the free health score's date-format check still awarded the document 10 of 10 points, because the odd formats happened to be internally consistent enough to pass a consistency check. A document can look fine on a format check and still not yield a timeline — which is why the benchmark measures recovery, not appearance.

Hazards compound: the heavily designed template fails on every axis at once

One corpus document combines the hazards the way real designed templates do — two-column table layout, contact only in the header, skills only as an image, creative headings, non-standard dates. It lost 38.5% of its canaries, filled 2 of 8 record fields, mapped 17 of its 20 extracted lines to no field, and scored 65 on the free health check against the control's 93 — same words, same person, same experience.

Structure is the only variable in this corpus. Every document except the two seeds carries literally identical content, so every gap between the control's numbers and a hazard document's numbers is attributable to structure alone.

The hazard that didn't bite (here): text boxes

We report the non-failures too. The text-box document — its skills list inside a floating drawing-canvas text box — lost 0 canaries on this parsing path: the extractor read the box's content. The damage was subtler: the box's text merged into the surrounding line and swallowed the Education heading that followed it, so the record's education field came up empty — but the skills themselves survived. Many extraction engines do skip text-box content entirely; ours, on this corpus, did not. A benchmark that only published the failures would be marketing — this line is what keeps the 100% figures above believable.

Full results

One row per corpus document. "Canaries lost" is verified content absent after extraction; "fields" is the parse record; "timeline" is date ranges the chronology parser recovered against the ranges the document carries; "health" is the free Resume Health Score the identical text receives.

What this means if you're applying

The corpus says structure fails in layers, and the layers feel identical from the outside. Content can vanish entirely (header contact, images), arrive but map to no field (creative headings), or arrive in the wrong order (columns and tables) — and in every case the candidate sees the same thing: silence. The fixes are equally concrete: contact details in the document body, conventional section headings, single-column flow, dates a parser can read, and nothing load-bearing carried as a graphic. You can check your own document against this exact parsing path for free — the Resume Health Score runs the same structure scan and record used above, and the ATS Simulator shows you the resulting record field by field, in your browser.

Methodology

The corpus. 12 .docx documents. Ten are generated with identical fictional content — one candidate, one career — varying only the structure under test: clean single-column control, table layout, two-column layout, contact in the page header, contact in the page footer, skills in a floating text box, skills as an embedded image, unusual section headings, non-standard date formats, and a compound document combining the hazards. Two are the pre-existing hazard test documents that seeded the corpus (a table-based resume and a two-column CV, different fictional candidates). Corpus, generator and ground-truth manifest are stored in the repository, and each file's hash is pinned.

Ground truth before measurement. Each document's manifest lists canary strings and where they were placed. Before anything is measured, the harness re-verifies every canary against the raw document XML — header and footer parts included — and rejects any canary claimed to sit outside the body that also appears in the body (so "lost from the header" can never be faked). Image-borne canaries are validated against the text file the image is deterministically rendered from.

The measurement. Each document runs through the same parsing path this site runs on real documents: the .docx text extraction used by every free tool and the paid audit pipeline (mammoth 1.12.0), the audit engine's document-structure scan, the simulator's field-mapping record, the audit engine's chronology parser, and the health-score composite with published weights. Canary survival is substring matching on the extracted text, whitespace-stripped and case-insensitive on both sides — a bias that runs toward survival, i.e. against overclaiming loss. The control document must come through perfectly or the harness refuses to emit results.

Reproducibility. The measurement is a pure function of the stored corpus and the pinned parser versions: re-running scripts/u15-run-benchmark.ts emits byte-identical results, the published page renders exclusively from that results file, and the test suite re-runs the full measurement from the stored corpus on every run — a drifted or hand-edited figure fails the build. Version 1.0, published 2026-08-14.

Keep reading

• The State of ATS 2026 — our first data study →
• How ATS Screening Actually Works →
• Free Sameness Score — the wording axis →
• Free Resume Health Score — test your own structure →