A single signal instead of per-provider rules
OpenAI, Google, Microsoft and Anthropic presented a joint proposal on October 30 to allow sites to object to the use of their content in the training of artificial intelligence models. The central element is a signal to be placed in the robots.txt file, designed to be recognized by all participating companies instead of requiring a separate instruction for each provider. AI4Business reports.
The proposal is called Machine-Readable Training Preference Signal and was filed as an Internet-Draft with the IETF by Kevin Kelley of Anthropic, in the AI Preferences working group. It is therefore a technical draft, not an approved standard. The new token, __usage_ai-training, indicates the purpose of use rather than a particular scanning software: to exclude the entire site from generative training, the draft provides for the lines User-Agent: __usage_ai-training and Disallow: /, with possible exceptions via Allow on individual folders.
Training and grounding: two distinct uses
The decisive technical point is the separation between training and grounding. In the first case, content contributes to modifying the parameters learned by a model; in the second, a system consults external information to construct or verify a response at the time of the request. A publisher may want to block training and accept grounding, because an AI response that links back to its site can increase visibility.
Microsoft announced two families of preferences: AI Training uses the common signal in robots.txt and can also be set per page; AI Grounding determines whether Bing, Copilot, Microsoft 365 and connected services may use a page to feed generated responses, via meta tags or an HTTP header. The options are Allow, the default when no choice is made; DisallowExtended, which limits use to experiences managed directly by Microsoft; and Disallow. According to the company, these choices do not in themselves change eligibility to appear in ordinary Bing results, but blocking grounding can reduce presence in AI experiences without automatic financial compensation.
Deadlines and differences to verify
Microsoft has set an operational sequence: from February 1, 2027, Bing will respect the new preferences as it scans sites, so uploading a rule does not guarantee immediate application. March 1, 2027 is the deadline for those currently using NoArchive or NoCache to limit training and grounding: these operators will have to publish the new preferences and, to continue limiting grounding, add them to pages and register a temporary choice in Bing Webmaster Tools, available from December 15, 2026 and valid until the crawler passes or in any case by September 30, 2027. This is not a general obligation for all sites.
Google, for its part, plans to interpret the common opt-out as a prohibition for Google-Extended when the operator has not given explicit instructions. Since Google-Extended also affects grounding in some Gemini experiences, a generalized opt-out could reduce visibility in Gemini Apps; it remains possible to add a specific choice for Google-Extended alongside it. The picture is therefore still evolving: the IETF draft is not a standard, the four groups have announced different commitments and timelines, and control is not active simultaneously across all of them. The old provider-specific rules will continue to be relevant, and the material does not yet clarify how cases not covered by the announced commitments will be handled.
