Scientific Paper: Scalable Watermarking for Identifying Large Language Model Outputs

In October 2024, Google DeepMind published the paper that would become the spine of Europe’s AI transparency regime. The field remembers it as the paper that made AI text detectable. Read the title again. It says scalable. It does not say robust.

The Paper

The abstract, verbatim (reference markers omitted):

Large language models (LLMs) have enabled the generation of high-quality synthetic text, often indistinguishable from human-written content, at a scale that can markedly affect the nature of the information ecosystem. Watermarking can help identify synthetic text and limit accidental or deliberate misuse, but has not been adopted in production systems owing to stringent quality, detectability and computational efficiency requirements. Here we describe SynthID-Text, a production-ready text watermarking scheme that preserves text quality and enables high detection accuracy, with minimal latency overhead. SynthID-Text does not affect LLM training and modifies only the sampling procedure; watermark detection is computationally efficient, without using the underlying LLM. To enable watermarking at scale, we develop an algorithm integrating watermarking with speculative sampling, an efficiency technique frequently used in production systems. Evaluations across multiple LLMs empirically show that SynthID-Text provides improved detectability over comparable methods, and standard benchmarks and human side-by-side ratings indicate no change in LLM capabilities. To demonstrate the feasibility of watermarking in large-scale-production systems, we conducted a live experiment that assessed feedback from nearly 20 million Gemini responses, again confirming the preservation of text quality. We hope that the availability of SynthID-Text will facilitate further development of watermarking and responsible use of LLM systems.

The Largest Blind Taste Test Ever Run

Here is the part of the paper everyone skips, and it happens to be the actual achievement. DeepMind did not test their watermark on a benchmark. They tested it on you. Roughly 20 million live Gemini responses, watermarked against unwatermarked, scored by real users pressing real thumbs. The thumbs-up rate differed by 0.01%. The thumbs-down rate by 0.02%. Statistically nothing. Twenty million people took part in the largest blind taste test in publishing history, and the result was that nobody could tell.

That is what the paper is about. Not catching machines - hiding the mark so well, and so cheaply, that there is no reason not to stamp everything. The word in the title is scalable because cost was the problem being solved. Strength was never on offer.

One Load-Bearing Idea

A language model produces a probability distribution over the next token, and a sampler picks one. Wherever several tokens are nearly equally likely, the sampler is spending randomness - and randomness is free real estate. SynthID-Text moves in there.

The scheme, which the paper calls Tournament sampling, works per token:

seed = hash(secret_key, last_4_tokens)        # H = 4, sliding window
candidates = sample 2^m tokens from the LLM   # m = 30, with repetition
for layer in 1..m:
    pair up the survivors
    each pair: winner = token with the higher g_layer(token, seed)
emit the last survivor

The g functions are pseudorandom scores derived from the seed - a private coin flip per token, per layer. The emitted token therefore leans toward high g-values, invisibly, one tournament at a time.

Three properties make this publishable rather than obvious:

• It is non-distortionary. Averaged over seeds, the output distribution equals the model’s own - the paper proves the single-token case and configures the system to preserve whole sequences. A repeated-context mask skips watermarking when the same context window reappears, so the scheme never warps text by insisting on the same choice twice.
• It is nearly free. On Gemma 7B-IT, 30-layer Tournament sampling raised per-token latency from 15.527 ms to 15.615 ms - 0.57%. The authors even built a watermark-aware variant of speculative sampling that provably preserves the acceptance rate, so the production speed tricks survive.
• Detection is just averaging. Score a text by the mean of its g-values across tokens and layers; watermarked text scores anomalously high. More text means more evidence. More entropy means more signal - which is the property to keep in mind for the next section.

And the authors say, in plain print, where it fails: paraphrasing weakens it, low-entropy output (exact code, memorized facts, short answers) barely carries it, and enforcing any of this on open-weight models “deployed in a decentralized manner is difficult”. The limitations section is not buried. It is candid, complete, and universally uncited.

For the mechanics in video form: How Watermarks Track AI Generated Content - a great explanation, as usual, from Computerphile.

Why It Runs Everything Today

Gemini shipped it in 2024. The EU’s Code of Practice on Transparency of AI-Generated Content asked signatories for interwoven, imperceptible marks from 2026-08-02. Anthropic adopted this exact method for Claude in August 2026. One sampling trick from one paper is now the de facto mechanism behind a continent’s content-transparency law - I took apart what that mark can and cannot prove separately.

The Verdict

The citation graph remembers a security paper. Nature 634 contains an economics paper: it made watermarking cost nothing - invisible at twenty-million-response scale, 0.57% latency, friendly to every production shortcut - and said out loud that it would not survive a paraphrase. The three failure modes the authors listed are the exact three ways it fails in the wild today.

The paper shipped a stamp and labeled it a stamp. The world filed it under locks - and then wrote laws that lean on the door.