OpenAI, Google, Microsoft and Anthropic move towards a common signal against unauthorized training

Anthropic, OpenAI, Google and Microsoft have declared their support for a proposed common standard based on the robots.txt file, the tool that websites have used for decades to indicate to search engines which pages can be crawled. The novelty concerns the extension of this mechanism to blocking the training of artificial intelligence models on content published online.

The adoption timelines stated by the four companies are not uniform. Anthropic has already announced support for the new signal, effectively making it operational on its side. OpenAI, on the other hand, plans to adopt it in the coming months, while Google and Microsoft indicate a longer horizon, set at 2027.

The mechanism, if it became a standard shared across the entire industry, would allow a publisher to express, with a single technical signal — rather than negotiating separately with each company — its wish to exclude the site’s content from the training of language models. This is a significant difference compared to the current situation, in which publishers often have to approach each artificial intelligence lab individually to obtain the exclusion of their texts.

The convergence of four of the leading companies in the sector on a single technical proposal marks a rare point of contact in an area — the training of models on copyright-protected material — that in recent years has more often seen the same players on opposing fronts with respect to publishers than in common positions among themselves.

It remains to be seen, within the timelines stated by each company — immediate for Anthropic, within a few months for OpenAI, by 2027 for Google and Microsoft — whether the standard will actually be implemented uniformly or whether the differences in schedule will, in practice, produce fragmented application in the meantime.