SoK: Watermarking AI-Generated Content — IEEE S&P Distinguished Paper#

The Authenticity Crisis#

The proliferation of generative AI has created an unprecedented authenticity crisis. Large language models produce text that is fluent, coherent, and often factually plausible — yet entirely synthetic. Image generators create photorealistic scenes that never existed. Audio synthesis clones voices with seconds of reference material. Video generation produces footage of events that never happened.

The consequences are already visible:

• Misinformation at scale: AI-generated news articles, social media posts, and commentary spread false narratives with the veneer of credibility that comes from naturalistic language and imagery.
• Identity fraud: Synthetic voices and images bypass biometric authentication systems, enabling financial fraud and unauthorized access.
• Intellectual property disputes: When AI can reproduce styles, voices, and creative patterns with high fidelity, determining originality and attribution becomes a legal quagmire.
• Trust erosion: As people become aware that any piece of content might be synthetic, trust in all digital media degrades — including authentic content.

Why Detection Alone Is Insufficient#

A natural first approach to identifying AI-generated content is post-hoc detection: training classifiers to distinguish synthetic from human-created content. The SoK paper explains why this approach is fundamentally fragile:

1. Adversarial arms race: Detection models can be defeated by adversarial perturbations, paraphrasing, or simply using a different generator that the detector wasn’t trained on.
2. Generalization failure: Detectors trained on content from one model family (e.g., GPT-4) often fail on content from another (e.g., Claude, Gemini), and degrade as models improve.
3. False positive costs: False accusations of AI authorship carry real harm — for students, journalists, and professionals whose legitimate work is flagged as synthetic.
4. Lack of attribution: Even when detection succeeds, it cannot determine which model generated the content or when it was generated.

Watermarking addresses these limitations by embedding a detectable signal at generation time, creating a proactive rather than reactive approach to content authentication.

Modalities Covered#

The SoK systematically analyzes watermarking techniques across five modalities:

1. Text: Watermarking the output of large language models (LLMs)
2. Image: Watermarking the output of diffusion models and GANs
3. Audio: Watermarking synthetic speech and music
4. Video: Watermarking generated or manipulated video sequences
5. Multi-modal: Watermarking content that spans multiple modalities (e.g., image-text pairs)

The Watermarking Pipeline#

The paper introduces a generalized watermarking pipeline that applies across modalities:

1
Generator → [Watermark Embedding] → Watermarked Content → [Distortions/Attacks] → [Watermark Extraction] → Detection/Verification

Each stage of this pipeline is analyzed in detail, with modality-specific variations clearly delineated.

Text Watermarking#

Text watermarking presents unique challenges because text is discrete — you cannot add imperceptible perturbations to a word in the way you can to a pixel. The SoK identifies several families of text watermarking approaches:

Image Watermarking#

Image watermarking has a longer history, but generative AI introduces new challenges and opportunities:

Audio and Video Watermarking#

Audio and video watermarking receive comparatively less attention in the literature, which the SoK identifies as a significant gap:

• Audio: Techniques adapted from music watermarking (e.g., AudioSeal) show promise for synthetic speech, but robustness against common audio processing (compression, noise addition, re-encoding) remains challenging.
• Video: Spatial-temporal watermarking extends image techniques across frames, but the computational cost of embedding and extracting watermarks from video streams at production quality is substantial.

1. Quality Preservation#

The watermark should not degrade the perceptual quality of the generated content. This is measured differently across modalities:

• Text: Perplexity, BLEU/ROUGE scores, human evaluation of fluency and coherence
• Image: FID, PSNR, SSIM, LPIPS, human evaluation
• Audio: PESQ, VISQOL, mean opinion scores

The SoK reveals a consistent finding: there is a fundamental tension between watermark strength and content quality. Stronger watermarks (more detectable, more robust) inevitably introduce greater quality degradation. The art of watermarking lies in navigating this tradeoff.

2. Detectability#

The watermark should be reliably detectable. Key metrics include:

• True positive rate (TPR): Probability of correctly identifying watermarked content
• False positive rate (FPR): Probability of falsely flagging human content as watermarked
• Area under the ROC curve (AUC): Overall detection performance
• Minimum text length / image size: The minimum content length needed for reliable detection

A critical finding: most published watermarking methods achieve excellent detection metrics (AUC > 0.99) under ideal conditions, but performance degrades significantly under realistic conditions with distortion and attacks.

3. Robustness#

The watermark should survive various transformations and attacks:

• Paraphrasing: Rewriting watermarked text while preserving meaning
• Translation: Converting watermarked text to another language
• Compression: JPEG, MP3, H.264 compression of images, audio, and video
• Cropping and resizing: Spatial transformations of images and video
• Noise injection: Adding random perturbations
• Adversarial attacks: Deliberately crafted perturbations designed to remove the watermark

The SoK’s systematic robustness evaluation reveals that most current watermarking schemes are not robust against determined adversaries. Paraphrasing attacks, in particular, are highly effective against text watermarking, while image watermarking is vulnerable to a combination of geometric transformations and noise.

4. Security#

Beyond robustness to generic distortions, security considers adversarial settings:

• Key extraction: Can an adversary recover the watermarking key by observing watermarked outputs?
• Spoofing: Can an adversary embed a watermark into human-created content?
• Removal: Can an adversary strip the watermark while preserving content quality?
• Forgery: Can an adversary modify an existing watermark to carry false information?

The SoK identifies security as the most underexplored property in the current literature, with many papers assuming the secrecy of the watermarking key without formal analysis of how it might be compromised.

5. Capacity#

How much information can the watermark carry? This ranges from:

• Zero-bit watermarks: Binary presence/absence detection
• Multi-bit watermarks: Encoding tens to hundreds of bits of metadata (model ID, timestamp, user ID)
• High-capacity watermarks: Encoding full messages, at the cost of quality and robustness

6. Efficiency#

The computational overhead of embedding and extracting watermarks:

• Embedding overhead: Additional latency introduced during content generation
• Extraction overhead: Computational cost of watermark detection
• Training overhead: For methods requiring model fine-tuning

The SoK finds that the most efficient methods (e.g., logit-bias text watermarking) add negligible overhead (<1% latency increase), while learning-based methods can require significant training investment but minimal inference overhead.

Removal Attacks#

These aim to remove the watermark while preserving content quality:

• Paraphrasing and rewriting: Using another LLM to paraphrase watermarked text is surprisingly effective against most text watermarking schemes
• Image-to-image translation: Running watermarked images through diffusion models to regenerate them
• Compression and degradation: Applying heavy compression, noise, and quality reduction
• Token substitution: Systematically replacing tokens in watermarked text with synonyms

Spoofing Attacks#

These aim to embed watermarks into content that wasn’t generated by the watermarked model:

• Key recovery: Observing many watermarked outputs to infer the secret key
• Masked diffusion attacks: Exploiting the structure of latent-space watermarks to inject them into non-generated images
• Signature copying: Extracting a watermark from one image and applying it to another

Forgery Attacks#

These aim to modify existing watermarks to carry false information:

• Bit-flipping attacks: Changing individual bits of a multi-bit watermark
• Collusion attacks: Combining multiple watermarked outputs to isolate and modify the watermark signal

Evasion Attacks#

These aim to generate content that avoids watermark detection:

• Non-watermarked models: Simply using a model without watermarking
• Partial regeneration: Mixing watermarked and non-watermarked segments
• Adversarial prompt engineering: Crafting prompts that exploit weaknesses in the watermarking algorithm

Finding 1: No Single Method Dominates#

The SoK’s comprehensive comparison reveals that no single watermarking method achieves the best performance across all properties simultaneously. Methods that excel at robustness often sacrifice quality; methods with strong security guarantees may have limited capacity; efficient methods may be vulnerable to specific attacks. This finding underscores the need for application-specific watermarking design.

Finding 2: Text Watermarking Is the Hardest Problem#

Among all modalities, text watermarking faces the most severe challenges:

• The discrete nature of text limits the information-theoretic capacity for embedding
• Paraphrasing attacks are simple, effective, and preserve meaning
• Translation to other languages often destroys text watermarks entirely
• The quality-robustness tradeoff is particularly acute: even small watermarking biases can measurably degrade text fluency

The SoK suggests that practical text watermarking may need to combine statistical methods with semantic fingerprinting or stylistic analysis.

Finding 3: The Security Foundation Is Shaky#

Most watermarking systems rely on the secrecy of a key, but the SoK identifies several scenarios where this assumption is questionable:

• API access: Adversaries with API access can query the watermarked model extensively, generating data for key recovery
• Open-source models: When the watermarked model is open-source, the adversary has full access to the watermarking algorithm
• Side-channel leakage: Watermark detection statistics may leak information about the key

The paper calls for watermarking systems that provide security beyond key secrecy, drawing on insights from cryptography (e.g., public-key watermarking, provable security guarantees).

Finding 4: Standards and Interoperability Are Missing#

The watermarking ecosystem is fragmented:

• Different providers use different watermarking schemes with no interoperability
• No universal detection infrastructure exists (unlike, say, digital certificates)
• Regulatory requirements (EU AI Act, state-level deepfake laws) mandate watermarking but don’t specify technical standards
• The absence of standards creates a collective action problem: individual providers watermark their content, but detecting watermarks from other providers is difficult or impossible

Finding 5: The Quality-Robustness Tradeoff Is Fundamental#

Across all modalities and methods, the SoK demonstrates that the tradeoff between content quality and watermark robustness is not merely a limitation of current techniques but appears to be information-theoretically fundamental. Any signal that modifies the content distribution to embed a watermark necessarily reduces the content’s fidelity to the unwatermarked distribution. The question is not whether this tradeoff exists, but where the Pareto frontier lies and how to push it outward.

For AI Model Developers#

1. Embed watermarking early: Retrofitting watermarking onto an existing model is harder than designing it into the generation pipeline from the start. Latent-space watermarking integrated during training produces the strongest results.
2. Plan for attack evolution: Any deployed watermarking scheme will be attacked. Design with key rotation, algorithm agility, and defense-in-depth in mind.
3. Measure quality impact rigorously: Don’t just measure detection AUC — evaluate the actual impact on content quality across diverse use cases, languages, and domains.

For Platform Operators#

1. Invest in detection infrastructure: Building watermark detection into content pipelines (upload filters, content moderation systems) is essential for making watermarking useful at scale.
2. Support multiple schemes: No single watermarking scheme will be universal. Detection infrastructure should be extensible to support new schemes as they emerge.
3. Combine with metadata: Watermarking is most powerful when combined with content provenance metadata (C2PA standard, digital signatures) to create a complete authenticity chain.

For Policymakers#

1. Avoid mandating specific techniques: Technology evolves rapidly. Regulations should specify outcomes (detectable AI-generated content) rather than specific watermarking methods.
2. Fund interoperability research: The biggest gap is not within any single watermarking scheme but between them. Funding for standards development and cross-scheme detection is high-impact.
3. Consider the global landscape: Watermarking requirements that only apply in one jurisdiction will be circumvented by generators operating elsewhere. International coordination is essential.

Provable Watermarking#

Current watermarking lacks formal guarantees. Can we develop watermarking schemes with provable bounds on false positive rates, robustness to specific attack classes, and information capacity?

Public-Key Watermarking#

Analogous to public-key cryptography, can we develop watermarking schemes where anyone can detect the watermark but only the key holder can embed it? This would enable universal detection without compromising security.

Cross-Modal Watermarking#

As multi-modal models (GPT-4V, Gemini) generate content spanning text, images, and audio, watermarking schemes that create coherent cross-modal signatures — where the watermark in the text component is linked to the watermark in the image component — could provide stronger guarantees.

Watermarking for Agentic Systems#

AI agents that take actions through tool use create new watermarking challenges: how do you watermark not just the text output but the sequence of actions, tool calls, and intermediate reasoning?

Adaptive Watermarking#

Watermarking schemes that adapt their strength based on the content’s sensitivity and the threat model. High-stakes content (e.g., news images, financial reports) could receive stronger watermarks, while casual content receives lighter treatment to preserve quality.