Our classifier scored “I have never seen a shadow figure” as a sighting

A post-mortem on embedding negation blindness in a production counting pipeline: how we found it, the fix that looked right and would have made things worse, the fix that shipped, and the probe set that now guards the door. August 2026.

Context, in three sentences

This site maintains a record of about 50,000 first-hand accounts of experiences people could not explain, and publishes counts like “the record holds 261 accounts of this phenomenon.” Those counts come from a two-stage classifier: sentence-level matching against exemplar phrasings and negative contrast examples using static sentence embeddings, then a contextual-encoder pass over the survivors. Every published number regenerates from a script, which is the property that made this bug findable and worth writing up.

How it surfaced

We ran a preregistered measurement on a Reddit thread of shared false-memory accounts (that experiment is here) and needed to classify each account as a memory of watching a film versus a memory of its packaging. The classifier kept scoring sentences like “I can’t for the life of me remember the actual movie” as viewing-memory evidence. It was reading the topic and ignoring the denial.

That raised the obvious question about the production pipeline. So we probed it: fourteen synthetic accounts whose only relevant sentence denies the phenomenon, one affirmative control per topic, across all nine production topics.

The result

Eleven of fourteen denial sentences were accepted as evidence. Every control passed, so this was not a broken classifier, it was a working classifier with a blind spot. The worst cases, with their evidence margins:

These are not marginal accepts. +0.629 is a stronger score than most genuine accounts earn. The embedding sees maximum topical overlap and no penalty for the one word that inverts the meaning. This weakness of dense embeddings is documented in the literature, but a documented weakness in general is different from knowing whether and where it bites your pipeline.

The fix that would have made it worse

The classifier already uses negative contrast examples: each sentence is scored best-similarity-to-exemplars minus best-similarity-to-negatives, so the reflex fix is to add the denials to the negatives list. That reflex is wrong, and it is wrong for the same reason the bug exists. The encoder cannot tell “I woke unable to move” from “I never woke unable to move,” so a negated negative cancels the genuine affirmative just as hard as it cancels the denial. You do not patch negation blindness with material the model cannot distinguish either.

The nuance that rules out the generic fix

The other reflex is a general negation detector that discounts any negated sentence. In this corpus that would be a disaster, because several phenomena are defined by negation. “I could not move” is the core of a sleep-paralysis account. “I heard my name and nobody was there” is the phenomenon itself. A classifier that discounts negation discounts the evidence.

What actually separates the cases is not negation, it is what is being negated: the experience versus something inside the experience. “I have never seen” denies the experience. “Nobody was there” is the experience.

What shipped

A denial gate: a deliberately narrow pattern for experiential denial and explain-away (“I have never seen,” “it was not the,” “turned out to be,” “nobody has ever”) with carve-outs for affirmative idioms like “never seen anything like it.” A sentence matching it is disqualified from serving as evidence. It runs before the argmax over sentence margins, so the account is not rejected, it just cannot be carried by a denial.

In the film-versus-packaging classification the gate is applied asymmetrically, because there a denial of watching is affirmative evidence for the packaging side. A gate with judgment about direction, not a blanket rule.

Results, including the boring one

After the gate: zero of fourteen probes leak, all nine controls still pass. Then we reclassified the full 48,728-account corpus, and the published counts came back byte-identical. The second stage and the certification vetoes had already been keeping denial-carried accounts out of the published numbers, so nothing we had told readers was wrong. We are publishing the near-miss anyway, because the first stage was one filter away from wrong and the place this gate genuinely matters is the live submission path, where real people write things like “I know it was not a dream” in every third account.

What we keep

The probe set is now a standing audit: any change to the exemplars, the gate, or the classifier has to run it clean before a number reaches a page, and every new topic gets denial probes as part of its certification. The broader doctrine, which this episode confirmed rather than taught us: our keyword layer and our embedding layer fail in opposite directions, keywords over-match surface and embeddings ignore polarity, so no published count rests on either alone.

The full method, including the parts of it that failed before this one, is at /method. The measurement that surfaced the bug is at /experiment/shazaam. Criticism is wanted and has a good track record here.