Navigating the Emerging AI Content Marketplace: A Guide for News Publishers
Returning from a week immersed in the AI ecosystem of San Francisco, publishers participating in a WAN-IFRA study tour gained a clearer understanding of a rapidly evolving market for news content. While the landscape of winners and losers is still taking shape, and uncertainties remain regarding demand, pricing, and content preferences, several key investments emerged as crucial for publishers seeking to capitalize on this developing market.
The AI content marketplace, once characterized as chaotic and dominated by unauthorized scraping and opaque pricing, is beginning to mature. As of early 2026, a functioning marketplace is taking shape, presenting both opportunities and challenges for news organizations.
Prioritizing Key Investments for Success
To effectively leverage this emerging market, news publishers should prioritize the following four areas:
- Managing automated bot traffic scraping.
- Cataloging and structuring content for delivery in machine-readable formats.
- Adapting to shifts in market demand, moving from generalized training data to specific data for fine-tuning or domain-specific applications, and grounding data for inference.
- Tracking the use of their content to understand how this market values different types of content and how it is priced.
1. Managing Automated Access
Controlling access to content by bots and scrapers is paramount. Ana Jakimovska, Head of AI strategy for Mediahuis, highlighted the ongoing issue of illegal content scraping, even behind paywalls, noting that her company blocks 100,000 bots and scrapers daily with minimal impact on overall traffic (WAN-IFRA).
Content Delivery Networks (CDNs) like Cloudflare, Akamai, and Fastly offer services to manage traffic and bot access. Historically, search engines crawled sites and, in return, sent traffic generating ad revenue. However, this dynamic is changing. Sam Else, Cloudflare’s Senior Director of Strategic Partnership for Media, Creators and AI, observed a shift in this quid pro quo (WAN-IFRA).
Data from Cloudflare’s Radar service reveals a decline in referrals from search engines. In 2026, Google returns one referral for every five crawls, Perplexity for every 155, and Anthropic for over 28,000 Cloudflare’s Radar service. This shift underscores the necessitate for publishers to negotiate terms and control access to their content.
People Inc.’s successful licensing deal with Microsoft demonstrates the power of controlled access. CEO Neil Vogel attributes their ability to secure the deal to using Cloudflare to control bot access.
Emerging protocols like Really Simple Licensing (RSL) and IAB’s Content Monetisation Protocols (CoMP) provide machine-readable licensing and payment instructions, complementing bot management services. RSL can even be integrated with CDN solutions.
2. Catalogue and Structure Your Content for Higher Returns
AI labs are increasingly recognizing the value of news media content. Brooke Hartley Moy, CEO and co-founder of Infactory, emphasized that AI progress is bottlenecked by data, not models or compute (WAN-IFRA). She noted that AI builders are actively seeking curated, annotated content, which she likened to journalism.
Companies like Infactory and Protege structure content for a cut of the revenue, while structured content can increase value by 10 to 30x. Publishers are now considering two distinct audiences: humans and machines, requiring different content delivery methods.
Madhav Chinnappa, formerly of the AP, BBC, and Google, stressed that structured data is now a “tablestake” for participation in the AI content market (WAN-IFRA).
3. Adapt to Shifting Demand from AI Labs
Several factors drive premium pricing for content, including rarity, clear IP control, quality, domain-specific focus, and continuity. The demand is shifting from general training data to specialized data for fine-tuning and inference.
The market is evolving towards high-value niche applications, exemplified by Perplexity’s focus on professionals in fields like finance, law, and sports. This trend is driving demand for domain-specific content and real-time grounding data – factual information used in answer engines.
4. Pricing for Training Data is Starting to Resolve
Pricing for content is becoming more precise, though still imperfect. Historically volatile, pricing for video content has increased 5 to 25-fold as demand shifts towards fine-tuning and specialized data. Deals facilitated by companies like Protege are generating significant revenue for content licensees.
The market is evolving into a multi-layered system, encompassing how data is used (training, fine-tuning, grounding), compensation models (pay-per-crawl, pay-per-use, long-term licensing), and a mix of bilateral deals and collective industry action.
Microsoft’s launch of its Publisher Content Marketplace and potential similar initiatives from Amazon signal a growing recognition of the value of news content. The Danish Press Collective Management Organisation and the newly formed Spur coalition (including The Guardian, BBC, Financial Times, and Sky News) demonstrate the power of collective action in negotiating with AI companies.
As Mediahuis’ Jakimovska stated, “We need to unite. That is where I consider we can win.” (WAN-IFRA).
Keep reading