AI Text Watermarking Under EU Rules: Why It Won't Stop Misinformation

The EU AI Act now requires AI companies to watermark AI-generated text, with a compliance deadline of 2 December. OpenAI and Anthropic have announced plans to add text watermarks to their models. But early evidence and expert warnings suggest these watermarks are far from reliable. For builders shipping AI products, the practical takeaway is clear: watermarking adds some friction for misuse but cannot be trusted as a definitive detector of AI-generated content.

How text watermarking works

Text watermarks rely on embedding a mathematically detectable pattern into the model's word choices. Large language models generate text by picking the most probable next token. By subtly biasing these choices, for example alternating between the most likely and second most likely word, a model can embed a pattern without changing the overall meaning. Detection tools then look for that pattern to identify AI-generated text. Anthropic has described its approach as using "low-stakes" choices during generation to leave an imperceptible signature.

Why watermarks are easy to evade

The core problem is that these patterns are fragile. James Padolsey, an AI safety researcher, created a tool called declaude that lightly edits AI-generated text to remove detectable patterns. He argues that even minor rewording, paraphrasing, or translation can strip the watermark. "It's not useful, and the way that people are going to try and use it will be incorrect," he told New Scientist. Open-source models that are not bound by EU rules will remain available, giving bad actors an easy way to generate unwatermarked content.

The false positive trap

Watermark detection tools are probabilistic, not definitive. Peter Scarfe at the University of Reading warns that these systems could flag text that was only processed by AI, such as a student using a model to check grammar on an otherwise human-written essay. Detection tools often include disclaimers that they should not be used to prove plagiarism. "If it says there's a 67.8 per cent chance that this text was generated in some way by AI, as an educator, how would I act upon that?" Scarfe said. False positives could create more problems than they solve.

What changes for builders

For teams building AI products, the main implication is that watermarking is a transparency measure, not a security or enforcement tool. It may create enough friction to slow mass disinformation campaigns, as Erman Ayday at Case Western Reserve University noted: "Anybody can tamper with the watermark if they spend enough time on it, but then, when you spend enough time on it, then maybe you should have just generated the content yourself." But for determined actors, evasion is trivial. Builders should not rely on watermarks for content moderation, plagiarism detection, or trust verification. Instead, treat them as one signal among many, and be aware that detection tools will produce false positives.

Caveats and unknowns

Several important limitations remain. Anthropic claims its watermarks do not affect output quality, but independent verification is lacking. The EU AI Act allows a grace period for already-deployed models, so not all text will be watermarked immediately. Detection tools themselves may not be widely available or reliable. And because open-source models can be run without watermarks, any regulatory benefit is limited to closed, API-based services. For builders, the safest approach is to assume that any AI-generated text can be stripped of its watermark with minimal effort.