A Sustainable AI Economy Needs Fair Data Deals: Position Paper from NeurIPS#

The modern AI revolution runs on data — mountains of it. Every large language model, every image generator, every recommendation system is built atop datasets assembled from billions of individual contributions. Yet the people who generate this data — the writers, artists, photographers, developers, and everyday internet users — see virtually none of the trillion-dollar valuations their labor makes possible. At NeurIPS 2025, Ruoxi Jia and colleagues presented a provocative position paper titled “Position: A Sustainable AI Economy Needs Data Deals That Work for Generators”, arguing that this asymmetry isn’t merely an ethical concern — it is a fundamental economic instability that threatens the long-term viability of the entire AI ecosystem.

This post breaks down the paper’s core arguments, explores its proposed framework for “data deals,” and examines the implications for AI security, governance, and policy.

The Unsustainable Status Quo#

Data as the Invisible Fuel#

Modern AI systems are insatiable consumers of data. GPT-class language models are trained on trillions of tokens scraped from the web. Image generation models like Stable Diffusion and DALL-E ingest billions of image-text pairs. Even specialized models for code, science, and medicine rely on massive curated datasets that ultimately trace back to individual human contributions.

The paper identifies a critical structural problem: the AI data economy operates on a one-way value extraction model. Data generators (creators, users, communities) contribute data — often without meaningful consent or awareness — and AI developers capture the resulting value. This creates what the authors call a “data commons tragedy” where:

Data generators are not compensated for the value their data creates
There is no feedback loop between data quality/value and the returns to generators
The incentive structure discourages future high-quality data generation
Legal and regulatory pressure is building toward an inflection point

Why This Matters for AI Sustainability#

The paper argues convincingly that this isn’t just a fairness issue — it’s a sustainability issue. Consider the cascading risks:

Data quality degradation: If generators have no incentive to produce high-quality, publicly available data, the quality of the data commons will decline over time. We already see this with the rise of AI-generated content polluting training datasets — a phenomenon researchers have termed “model collapse.”
Regulatory disruption: The EU AI Act, evolving copyright jurisprudence (e.g., the New York Times v. OpenAI case), and national data sovereignty laws are all moving toward requiring meaningful consent and compensation for data use. Companies that don’t plan for this will face existential legal risk.
Market concentration: When data is extracted for free, only the best-funded companies can build competitive models. This concentrates power in a handful of corporations, reducing innovation and creating systemic risk.
Social license erosion: Public trust in AI is already fragile. The perception that AI companies are exploiting creators’ work without compensation fuels backlash that could lead to restrictive regulation that stifles the entire field.

The Concept of “Data Deals”#

The heart of the paper is its proposal for structured data deals — formal arrangements between data generators and AI developers that ensure fair value distribution. The authors draw parallels to existing economic frameworks like licensing deals in the music industry, royalties in publishing, and revenue-sharing agreements in platform economics.

Principles of Fair Data Deals#

The paper proposes several foundational principles that any viable data deal framework must satisfy:

1. Transparent Valuation#

Data must be valued in a way that is transparent, auditable, and grounded in measurable impact. The authors point to techniques from cooperative game theory — particularly the Shapley value — as one approach to attributing marginal value to individual data contributions. While computing exact Shapley values at scale remains impractical, approximation methods and sampling-based approaches can provide reasonable estimates.

The paper also explores data valuation benchmarks that measure how much a particular dataset or data point improves model performance on specific tasks. This creates a quantifiable basis for negotiation between generators and developers.

Current data collection practices rely heavily on take-it-or-leave-it terms of service that provide no genuine choice. The paper argues that data deals must include:

Granular consent mechanisms that allow generators to specify exactly how their data can be used (e.g., for specific model types, specific domains, or specific time periods)
Revocation rights that allow generators to withdraw their data from future training runs
Collective bargaining mechanisms that allow generators to negotiate as groups rather than individuals, addressing the massive power asymmetry between individual creators and AI companies

3. Fair Compensation Models#

The paper outlines several compensation models that could form the basis of data deals:

Lump-sum payments: Fixed payments for dataset access, similar to stock photography licensing
Revenue sharing: Ongoing payments tied to the commercial success of models trained on the data
Usage-based payments: Compensation proportional to how frequently or significantly a generator’s data influences model outputs
Hybrid models: Combinations of the above, potentially with minimum guarantees and performance bonuses

4. Dispute Resolution and Enforcement#

Any data deal framework must include mechanisms for resolving disputes between generators and developers. The paper suggests:

Independent auditors who can verify that data is being used within agreed-upon terms
Technical enforcement mechanisms like watermarking and provenance tracking that make unauthorized use detectable
Arbitration frameworks modeled on existing intellectual property dispute resolution systems

The Technical Infrastructure for Data Deals#

One of the paper’s most valuable contributions is its discussion of the technical infrastructure needed to make data deals practical at scale. This is where the intersection with AI security becomes most apparent.

Data Provenance and Tracking#

For data deals to work, we need robust systems for tracking where data came from, who generated it, and how it has been used. This requires:

Cryptographic provenance: Using digital signatures, hash chains, or blockchain-based systems to create tamper-proof records of data origin and lineage
Watermarking: Embedding identifiable markers in data (text, images, audio) that survive transformation and allow tracing back to the original generator
Model auditing tools: Techniques for determining which training data influenced specific model behaviors or outputs, enabling usage-based compensation

Privacy-Preserving Data Markets#

The paper explores how federated learning, differential privacy, and secure multi-party computation could enable data deals where generators contribute to model training without revealing their raw data. This is particularly important for sensitive domains like healthcare, finance, and personal communications.

In a federated data deal, generators could:

Retain physical control of their data on their own devices
Contribute gradient updates to a shared model
Receive compensation based on the measured improvement their updates provide
Maintain privacy guarantees through differential privacy mechanisms

Data Marketplaces and Platforms#

The paper envisions data marketplaces — platforms that facilitate data deals at scale by providing:

Standardized data valuation tools
Template agreements and contracts
Dispute resolution services
Technical infrastructure for data transfer, tracking, and auditing

Several startups and research projects are already exploring this space, including DataXhange, Ocean Protocol, and Scale AI’s data marketplace, though the paper notes that none yet provide the full suite of capabilities needed for truly fair data deals.

Connections to AI Security#

While the paper is primarily an economics and policy contribution, it has significant implications for AI security that deserve attention.

Data Provenance as a Security Primitive#

The same provenance tracking infrastructure needed for data deals is also critical for AI supply chain security. If we can trace every training data point back to its source, we can:

Detect data poisoning attacks where adversaries inject malicious data into training sets
Verify data integrity by confirming that training data matches its provenance records
Enable targeted remediation when compromised data is identified, rather than requiring complete retraining

Economic Incentives for Data Quality#

Fair compensation creates natural incentives for data generators to maintain and improve data quality. This is a form of economic defense against the data quality degradation that makes models more vulnerable to attacks and less reliable in deployment.

When generators are compensated based on the quality and usefulness of their data, they have direct financial incentives to:

Produce accurate, well-labeled data
Maintain data freshness and relevance
Resist attempts by adversaries to corrupt their data contributions

Reducing the Attack Surface of Data Collection#

Current mass-scraping approaches to data collection are inherently vulnerable to manipulation. When data is collected indiscriminately, it’s easier for adversaries to inject poisoned data. A data deal framework that involves direct relationships between generators and developers creates a more controlled and auditable data pipeline.

Challenges and Open Problems#

The paper is refreshingly honest about the challenges its proposal faces:

The Free-Rider Problem#

Even if major AI companies adopt data deals, there will always be actors (in less regulated jurisdictions, or operating illegally) who continue to scrape data without compensation. This creates a competitive disadvantage for compliant companies — unless regulatory frameworks create a level playing field.

Valuation at Scale#

Accurately valuing individual data points within a training dataset of billions is computationally and conceptually challenging. While the paper discusses approximation methods, this remains an active area of research. The authors note that perfect valuation is not necessary — what matters is that the valuation is good enough to create fair-enough deals that sustain the ecosystem.

Collective Action Among Data Generators#

Individual data generators have almost no bargaining power against major AI companies. The paper discusses the need for data cooperatives, unions, or collective licensing organizations (similar to ASCAP or BMI in the music industry) that can negotiate on behalf of generators at scale.

Technical Feasibility#

Many of the technical infrastructure components — particularly provenance tracking and model auditing — are still in early research stages. Significant investment in research and development will be needed before they can support data deals at the scale required by modern AI training.

International Coordination#

AI development is a global industry, but data regulation is inherently local. The paper acknowledges that effective data deals will require some degree of international coordination, perhaps through treaty frameworks or mutual recognition agreements similar to those used in intellectual property law.

Implications for AI Governance#

The paper’s arguments have direct implications for AI governance frameworks being developed by governments worldwide:

NIST AI Risk Management Framework#

The NIST AI RMF’s emphasis on trustworthy AI implicitly requires fair treatment of data generators. Incorporating data deal principles into the framework’s guidance on data governance could provide a practical path to implementation.

EU AI Act#

The EU AI Act’s requirements for data governance and transparency create a natural regulatory home for data deal requirements. The paper suggests that the Act’s implementing guidelines could include specific provisions for data provenance, consent, and compensation.

Copyright and Intellectual Property Reform#

The paper argues that existing copyright frameworks are insufficient for the AI context. The unique characteristics of AI training — particularly the transformation of individual works into statistical patterns — require new legal frameworks that go beyond traditional copyright. Data deals could serve as a market-based solution that reduces the need for litigation and regulatory intervention.

Practical Recommendations#

For organizations building AI systems today, the paper offers several practical recommendations:

Start tracking data provenance now: Even if you’re not yet compensating generators, building provenance tracking into your data pipeline will be essential when data deals become standard.
Engage with data generators proactively: Rather than waiting for legal requirements, reach out to the communities whose data you use and explore voluntary data deal arrangements. This builds goodwill and positions you ahead of regulatory curves.
Invest in valuation research: Support academic and industry research into data valuation methods. The tools that emerge from this research will be the foundation of the data deal economy.
Design for data deal compatibility: Build your data infrastructure with the expectation that you will need to track, value, and compensate for data usage. This means investing in metadata management, version control for datasets, and audit capabilities.
Participate in standards development: Organizations like NIST, ISO, and the W3C are beginning to develop standards related to data provenance and AI governance. Active participation ensures that emerging standards are practical and implementable.

Conclusion#

The position paper from Jia et al. makes a compelling case that the current AI data economy is structurally unsound. Without fair mechanisms for compensating data generators, the AI ecosystem faces declining data quality, regulatory disruption, and social backlash that could undermine decades of progress.

The proposed data deal framework is ambitious but grounded in practical economics and existing technical capabilities. It doesn’t require a complete overhaul of the AI industry — rather, it proposes a gradual transition toward a more sustainable model that benefits all participants.

For AI security professionals, the message is clear: data provenance and fair compensation are not just economic or ethical concerns — they are security requirements. A data economy built on extraction and exploitation is inherently fragile. Building one based on fair deals and mutual benefit creates a more resilient, more secure, and more sustainable foundation for the AI systems of the future.

As the AI industry matures and regulatory frameworks solidify, the ideas in this paper are likely to move from academic position to practical necessity. Organizations that begin preparing for the data deal economy today will be better positioned — both competitively and ethically — than those that cling to the unsustainable practices of the past.

Paper Reference: Ruoxi Jia et al., “Position: A Sustainable AI Economy Needs Data Deals That Work for Generators,” NeurIPS 2025. Available at NeurIPS Virtual.

Newsletter