Tests for a PDF · Ata Kuyumcu's Blog

My CV has a test suite. This is either reasonable or a symptom.

The excuse is that I’m not the first reader. An applicant tracking system is, and it wants text. Every visual decision in the document (the font, the margins, where the dates sit) is invisible to it. The only thing it can see is whatever pdftotext scrapes off the page.

So the resume is a Typst document, make compiles it into dist/, and make test runs a Python script over every PDF it finds. Four checks:

The file starts with the bytes %PDF-.
pdfinfo reports a non-empty Author and Title.
It has at least one page.
pdftotext gets at least 200 characters out of it.

The first three are cheap paranoia. The fourth is the one I actually wanted. A PDF can be a picture of a document. If the text layer is broken or absent the file looks perfect to me and is blank to every parser between me and a human, which for a CV is most of the audience. 200 is an arbitrary number that separates “there is text in here” from “there is not”.

$ make test
Checking 1 PDF(s)...

  dist/resume/Ata Kuyumcu - CV.pdf
         Author: Ata Kuyumcu
         Title:  Ata Kuyumcu
         Pages:  2
         Text:   5054 chars

All 1 PDF(s) OK

5054 characters. There is text in there.

Writing for the parser

The tests check the output. The template tries to make the output easy to read in the first place: single column, no tables, half-inch margins, tech stacks as comma-separated plain text rather than icons. A two-column layout is the classic way to hand a parser your job titles interleaved with your dates.

And this, near the top of the template:

set text(
  font: font,
  size: font-size,
  lang: lang,
  // Disable ligatures so ATS systems do not get confused when parsing fonts.
  ligatures: false,
)

That comment is the kind of thing you read on resume advice sites. Ligatures turn “ffi” into one glyph, the theory goes, and the parser hands the recruiter “oce” instead of “office”. I believed it enough to write it down. I never checked it.

So I checked it. Two files, identical except for one boolean:

#set text(font: "Charis SIL", ligatures: true)
Office workflow efficiency. Certified affiliate. Final draft.

That line is 61 characters. With ligatures on, Typst writes 53 glyphs into the content stream. With them off, 61. The ligatures are real and they are doing exactly what the folklore says they do, collapsing eight characters into four glyphs.

Then I extracted both:

$ pdftotext lig-on.pdf -
Office workflow efficiency. Certified affiliate. Final draft.

$ pdftotext lig-off.pdf -
Office workflow efficiency. Certified affiliate. Final draft.

Identical. Because both PDFs carry a ToUnicode CMap, which is the table that says “glyph 0x0002 means the letters f, f, i”. Typst writes one either way. pdffonts has been telling me this the whole time in a column I never read:

name                       type          encoding    emb sub uni
-------------------------- ------------- ----------- --- --- ---
PYRJVB+CharisSIL           CID TrueType  Identity-H  yes yes yes

uni yes. The glyphs map back to Unicode.

I kept the setting anyway, which I want to be honest about rather than dress up. It protects against an extractor that ignores ToUnicode, or a producer that doesn’t write one. Both exist. I just can’t name the specific ATS that gets this wrong, and neither can any of the pages that gave me the advice. That’s the whole problem with this genre: the failure is real in principle, unobservable in practice, and the mitigation is free, so everyone does it and nobody measures it. I’m now one of the people who measured it and did it anyway.

It isn’t quite free. Disabling ligatures also kills ---, so every date range in the document goes the long way around:

// Cannot just use normal --- ligature because ligatures are disabled for good reasons
start-date + " " + sym.dash.em + " " + end-date

A defensive setting against a parser I’ve never met, and the cost is that I can’t type an em dash normally.

The check that never ran

While I was in there I found this:

EXPECTED = {
    "dist/resume/main.pdf": {
        "author": "Ata Kuyumcu",
        "title_contains": None,  # just check non-empty
    },
}

relative = str(pdf_path)
if relative in EXPECTED:
    exp = EXPECTED[relative]
    if exp.get("author") and exp["author"] != author:
        fail(pdf_path, f"expected author '{exp['author']}', got '{author}'")

The build produces dist/resume/Ata Kuyumcu - CV.pdf. The dictionary is keyed on dist/resume/main.pdf, which was the filename before I renamed the output so that recruiters would get something better than main.pdf in their downloads folder. relative in EXPECTED has been False on every run since.

I confirmed it the dumb way, by changing the expected author to somebody else and running the suite:

$ sed 's/"author": "Ata Kuyumcu"/"author": "Somebody Else Entirely"/' ...
All 1 PDF(s) OK

Green. The suite will not tell me if my CV is by Somebody Else Entirely.

That was the only check in the file that asserted anything about which document this is. The other four describe a file. Valid header, some metadata, some pages, some text: a PDF of your CV passes all four of mine. The one assertion that knew whose name was supposed to be on it died silently to a rename, and the tests kept printing Author: Ata Kuyumcu on the line right above the check that wasn’t comparing it to anything.

The font, briefly

The same shape of bug got me in CI. The templates ask for Charis SIL, the runner never installed it, and Typst treats a missing font family as a warning and exits 0. So CI published a perfectly valid PDF in the fallback serif for three weeks, and all four checks were delighted with it. The fix is a font install plus one line:

pdffonts "dist/resume/Ata Kuyumcu - CV.pdf" | grep -q CharisSIL

The version is pinned to 6.101 because Charis 7 renames the family to plain “Charis”, and Typst resolves fonts by family name. A newer, better version of the correct font would produce an identical silent fallback.

What this suite is actually for

Four checks that describe a file, one that describes the document and doesn’t run, one grep for a font, and a template full of defensive settings against a reader I have never observed. Put like that it’s not a great scoreboard.

But the 200-character check is still the one I’d keep if I could only keep one, and it’s still the one most likely to fire, because “the text layer broke” is a thing that actually happens to PDFs and it’s invisible on screen. The rest is me guessing at what a parser wants and occasionally, as with the ligatures, being able to prove I guessed wrong and doing it anyway.

I did fix the author check, though not by correcting the key. Fixing the key would have left the shape of the bug in place: a lookup that matches nothing does nothing, quietly, and the next rename starts the clock again. It’s now a constant compared against every PDF under dist/, which has no lookup to miss.

if author and author != EXPECTED_AUTHOR:
    fail(pdf_path, f"expected author '{EXPECTED_AUTHOR}', got '{author}'")
    ok = False

The way I know it works is that I poisoned the constant and the suite went red, which is the check I should have run on the original.