Retrieval does not throw when it is wrong

A service that is down tells you it is down. Retrieval does not. When retrieval is wrong the model still answers, in the same confident register it uses when it is right, and the only signal is a customer who is quietly misinformed.

Both of the bugs below were in production. Neither threw. Neither appeared in an error rate. I want to write them down because they changed how I think about the whole retrieval layer.

A bank asked Lira what a customer needs in order to open an account. Two crawled pages from the public website came back scoring 0.51 and 0.45. The bank's own written policy document scored 0.41. Ranking on similarity, the website won, and the answer was confidently wrong.

Nothing was broken. The website pages genuinely were more semantically similar to the question, because marketing copy is written in customer language and policy documents are not. Similarity was measuring the wrong thing, and measuring it correctly.

The obvious fix is to weight documents by trustworthiness. I tried it and stopped. To close a 0.10 gap you need a bonus large enough that it starts floating barely relevant policy documents above highly relevant ones, and once two numbers are being added together nobody can explain why a given answer was chosen.

So authority became a precedence rather than a weight. Sources sit in tiers. The highest tier with a real match answers the question, and lower tiers are not consulted at all. Answer from policy if policy has anything to say; fall back to the website only when it does not. That rule is one sentence, a support lead can understand it, and it cannot be tuned into nonsense by a coefficient.

Workspaces can tag knowledge-base sources into segments, so a session about one product does not retrieve documentation for another. The filter was the obvious one: match any of the active segments.

The first workspace that switched segmentation on lost everything. Every document indexed before tagging existed carried an empty segments array, and a match-any clause matches neither an empty array nor a missing field. The filter was not wrong about the documents it excluded. It excluded all of them.

What made it dangerous is the failure mode. There was no error, no empty-result alarm and no drop in traffic. The agent simply started saying it did not have enough information, in fluent and apologetic prose, which is exactly what a well-behaved agent says when a knowledge base genuinely lacks an answer. It looked like correct behaviour.

The fix is small: the clause is now a should over the segment match and an is_empty check, so untagged sources stay reachable unless the workspace explicitly opts into strict mode. The lesson is not small. A filter that silently narrows to nothing is indistinguishable, from the outside, from a system working perfectly on a hard question.

Neither was catchable by the tools we normally reach for. There was nothing to put in a try/catch, nothing to alert on, and nothing a unit test of the filter function would have flagged, because the filter did precisely what it said.

What catches them is measuring retrieval as retrieval: recall against a set of questions whose right answers are known in advance. Recall at 5 goes to zero the moment a filter hides the corpus. It also catches the authority problem, because a golden row can say that the policy document is the right source and the marketing page is not.

That is the actual reason I built an evaluation harness, and it is why its retrieval suite includes rows where the correct outcome is no results at all. A retriever that always returns something is the failure that reads as success.

I would have written the golden dataset before the retrieval features, not after. Every feature that filters or reranks is a chance to silently narrow the corpus, and each one shipped for a while with no way to see it happening.

I would also track the rate of I-do-not-have-enough-information answers per workspace as a first-class metric. Both bugs would have shown up as a step change in that number days before anyone reported them, and neither was visible in error rates, latency or traffic.