Detecting statistical LLM watermarks
Generation-time text watermarks work by introducing a secret, keyed relationship between preceding context and token selection. At each position the decoder subtly prefers tokens consistent with a hidden rule. A detector that knows the rule recomputes it over observed text and aggregates weak evidence across many positions.
How detection accumulates evidence
In the canonical Kirchenbauer green-list construction, a fraction γ of the vocabulary is designated “green” for each context. The detector counts green tokens and compares against the expected base rate. The statistic grows with usable token count — longer, higher-entropy passages generally provide more observations.
Production systems such as SynthID Text use different sampling rules (Tournament Sampling) but the principle is similar: aggregate correlation with keyed scores rather than visible markers.
False positives and false negatives
Statistical detectors operate at chosen thresholds. A low false-positive rate is essential at scale — even 0.1% specificity produces enormous numbers of incorrect flags across billions of documents. Short text, low-entropy passages, mixed human/AI documents and heavily edited content all increase false-negative risk.
A serious detector should expose calibrated evidence — a score, confidence band or p-value — rather than a binary “AI / human” verdict without context.
LLM watermarking primer → · Kirchenbauer / KGW guide → · SynthID Text →