Jobhunter: Retrieval was the easy half

A job-search agent that scored roles with hybrid retrieval, then learned the expensive way that a plausible email address is not a contact. One in four inferred addresses delivered; published ones, nine in ten.

Solo — retrieval, scoring, contact resolution · 2026

A note on who is reading this. Some of you received a message from this system, or one much like it. The write-up is not about volume — the volume was the mistake. It is about what the delivery data said about guessing, and what the system does differently now: fewer messages, only to addresses somebody published, about roles the retrieval could defend.

The problem

Matching a person to a role is a retrieval problem wearing a generation problem's clothes. Keyword search misses the job that describes the same work in a different vocabulary — "agent engineer" and "applied AI, tools and orchestration" are the same posting to a human and two unrelated documents to an index. Embeddings alone have the opposite failure: they will happily rank a neighbouring role that shares the theme and none of the hard requirements.

So the scoring is hybrid. The lexical side handles the requirements that are effectively binary — the stack, the location, the seniority, the eligibility — and the dense side handles the paraphrase. Fused, the lexical side gates and the dense side ranks: a posting that fails a hard requirement never makes the shortlist no matter how well it reads, and everything that passes is ordered by how close the work actually is.

Getting that right felt like the hard part of the project. It was not. The hard part was one field: the address to send to.

Architecture

Retrieval on the left is the part that worked. The gate on the right is the part that had to be added.

Job postings from boards, company pages and feeds — normalized and deduped
Lexical match: the literal requirements
Dense match: the same job, described differently
Fused score — lexical gates, dense ranks
Shortlist: roles worth a message
Contact resolution: a published address, or one inferred from a pattern
Verification gate: is this address actually deliverable?
Send one message, written to the posting — or route to a manual queue
Delivery ledger: delivered, bounced, replied — measured by source, not by send count

Key decisions

Lexical gates, dense ranks

The two halves of the score do different jobs and are not averaged into mush. Anything binary — does this role want the stack I have, is it in a place I can work, is the seniority in range — is checked literally and disqualifies on failure. The dense side never rescues a posting that failed one of those; it only orders what is left. A hybrid score that lets a beautiful embedding match override a hard requirement is just an embedding score with extra steps.

An inferred address is a hypothesis, not a contact

`first.last@company.com` is a guess with the shape of data, and every downstream system treats it as fact. That was the actual defect: the pipeline had no type distinction between an address a human published and an address the system invented from a pattern, so both flowed into the same send queue with the same confidence.

Measure delivery, not sends

The instrumentation that changed the system was not a better scorer, it was a ledger that recorded delivered, bounced and replied, grouped by how the address was obtained. Pattern-inferred addresses delivered about a quarter of the time. Addresses published on the posting or the company site delivered 90–97%. Nothing about the retrieval had any effect on either figure — which is the whole point. Three out of four inferred sends reached nobody, while spending the sending domain's reputation to do it.

Put the gate between the inference and the action

Inferred addresses no longer get sent to. They go to a queue where a published address is found or the role is skipped, and the retrieval score decides whether a role is worth that manual minute at all. Volume fell hard. Delivery per attempt went up, and so did the share of messages that were about a role the scorer could actually justify.

The lesson generalizes past email

Anything an agent infers — a guessed identifier, a synthesized fact, a plausible address — needs a verification step between the inference and the irreversible action, and the verification has to be measured separately from the thing that produced it. Retrieval quality has exactly no effect on an address that does not exist, and a pipeline that reports sends will never tell you that.

Numbers

~25% — delivered — pattern-inferred addresses (the rest bounced or vanished silently)
90–97% — delivered — published addresses (on the posting, or on the company site)
3 in 4 — inferred sends that reached nobody (paid for with sender reputation)
1 — gate added (verification between the inference and the send)

These are delivery rates from my own runs over a few hundred messages, not a controlled study — the sample is one sender, one domain, one narrow slice of a market. The direction is not subtle enough for the sample size to matter much, but treat the specific percentages as a log, not a benchmark. The outcome I would defend is the design change, not the decimal.