Farewell to Free Crawling: How RSL Protocol is Rewriting the Rules for Publishers and AI

Content Publishers Counter AI with the New RSL Protocol: Is It Too Late for Content Licensing?

The New Internet Standard: The RSL Protocol offers an innovative solution to the problem of web content exploitation, which may give human creators a real chance to compete in the growing AI economy.

Summary of ZDNET's Key Points on the RSL Protocol

  • Media companies announced: A new web protocol: RSL.
  • RSL aims: To put publishers back in the driver's seat.
  • The RSL Collective will attempt: To set content prices.

AI companies continue: To collect and extract massive amounts of digital content from websites. In response, a group of leading publishing and technology companies – such as Reddit, Yahoo, People, O'Reilly Media, Medium, and Ziff Davis (ZDNET's parent company) – developed a new licensing standard known as Really Simple Licensing (RSL).

The RSL protocol can be seen: As a stricter, more specialized version of Really Simple Syndication (RSS). While RSS focuses on publishing and distributing content across the web, RSL sends a clear message to AI crawlers: "You will no longer be able to consume my content for free."

Also: AI doesn't "infer" at all - How this team exposed industry hype

The core idea of the RSL protocol lies: In its simplicity and effectiveness. Instead of relying on the traditional robots.txt file, which AI bots often ignore, publishers can now embed machine-readable licensing terms within their content.

Does the publisher want attribution?: It can be specified. Do they prefer to be paid for every crawl operation performed by AI robots on their content, or even for every time an answer is generated based on their article? Yes, the RSL protocol also provides the necessary tags to specify these conditions.

This flexible approach enables: Publishers to clearly define the terms of use for their content: Is it available for free crawling, requires a paid subscription, or is subject to a "per inference" cost? This means paying fees every time AI models like ChatGPT, Gemini, or others, use the content to generate responses.

Key Advantages and Capabilities of the RSL Protocol for Content Licensing

  • Standardized licensing vocabulary: Allows publishers to clearly define licensing and compensation terms, including options for free attribution, attribution requests, payment for each AI crawl operation, or payment for each generated inference.
  • Automated content licensing: An open protocol that facilitates automated content licensing processes, creating an efficient licensing environment across the internet between content owners and AI companies.
  • Unified catalogs: Provides mechanisms for creating unified public catalogs of licensable content and datasets, based on RSS standards and Schema.org metadata.
  • Digital asset encryption: An open protocol for encrypting digital assets, ensuring secure licensing for private and non-public content, such as paid articles, e-books, exclusive videos, and specialized training datasets.
  • Collective licensing support: Supports collective licensing through organizations like the RSL Collective or any other licensing server compliant with RSL standards, to enhance negotiation power.

The RSL protocol is: An innovative solution to a complex problem facing content publishers. Tim O'Reilly, CEO of O'Reilly Media and a key supporter of the RSL initiative, emphasizes its importance by saying: "While RSS was vital to the evolution of the internet, the current situation where AI systems consume and reuse content without permission or compensation requires an evolution of the rules. RSL represents this necessary evolution."

O'Reilly is right: The RSS protocol contributed significantly to the web's expansion in its early stages, from blogs to news publishing and podcasts. But today's web landscape is no longer limited to competing for human attention; it has become an arena for providing the data necessary to train and infer AI models, which often benefit from this content without offering compensation to the sites from which they drew their power.

While technology forms: The cornerstone, commercial aspects remain crucial. This is where the "RSL Collective" comes in. Similar to music rights management organizations like ASCAP and BMI, this non-profit organization acts as a central clearinghouse to manage the rights of publishers and creators. Creators can join for free, pool their rights, and allow the collective to negotiate with AI companies to ensure they receive fair compensation.

Also: DeepSeek may change the world of AI again - What we know

As is known in the publishing industry: Independent writers or even most individual media organizations lack sufficient leverage to confront giants like OpenAI or Google. But when a group representing "millions" of online creators joins forces, it gains tremendous negotiating power. (Disclosure: In April 2025, Ziff Davis, ZDNET's parent company, filed a lawsuit against OpenAI, alleging copyright infringement in the training and operation of its AI systems.)

Historical Context: How the Need for the RSL Protocol Arose?

Let's review the context: That led to the emergence of this protocol. Over the past years, AI systems have extensively exploited internet content, benefiting from it being an "open buffet" without licensing costs. This model was previously possible when web economies primarily relied on advertising. However, this landscape has drastically changed. The old advertising model exhausted publishers, while generative AI companies accumulated billions of dollars in funding.

Also: 5 ways to bridge the AI skill gap in your business

Therefore, the RSL protocol seeks: To integrate a comprehensive licensing framework directly into the core web infrastructure. As an open-source protocol, similar to RSS, RSL is available to everyone. From large platforms like Yahoo to niche recipe bloggers, RSL empowers web publishers to specify the required compensation terms when AI systems crawl their content.

The RSL protocol's technical steering committee: Is responsible for its development and direction. This committee includes a selection of prominent web protocol engineers, including: Eckart Walther (co-author of RSS), R.V. Guha (developer of Schema.org and RSS), Tim O'Reilly, Stefan König (Yahoo), and Simon Westow (Fastly).

The web has always been built: On fundamental, invisible standards such as HTTP, HTML, RSS, and robots.txt. In the Web 1.0 era, "social contracts" were written directly into the code structure. If the RSL protocol achieves widespread adoption, it may represent the next evolutionary layer in this sequence: The layer that finally provides human creators with a real and tangible opportunity to compete in the AI economy.

And the RSL protocol may contribute: Thanks to its mechanisms, to ending the exploitation of content by AI, to terminate what resembles an "open buffet" that leaves no room for original creators.

Next Post Previous Post
No Comment
Add Comment
comment url