Website operators may soon be able to express a single AI training preference to several major model providers. Google, Microsoft and OpenAI each announced support on October 30 for a shared signal in robots.txt, based on work at the IETF. This is a planned implementation of a draft approach, not an immediate universal rule: companies have different adoption schedules and existing crawler controls remain relevant.

Google says a shared policy token will let publishers communicate a broad training preference without maintaining a separate opt-out for every participating provider. If the token disallows training and the site has not explicitly configured Google-Extended, Google intends to interpret it as a Google-Extended disallow. Site-specific Google-Extended choices will still take precedence where specified. Google targets 2027 for support, and notes that sites explicitly allowing Google-Extended can remain eligible for Gemini Apps responses that include source links.

Microsoft's approach adds a distinction that matters for publishers: training a generative model is not the same operation as grounding an AI-generated answer in a live web page. Its Bing implementation supports training preferences in robots.txt and on individual pages, while grounding choices are page-level. The DisallowExtended option permits grounding within Microsoft-operated experiences but not through Microsoft grounding services supplied to third parties.

Microsoft says it will begin honoring the new preferences on recrawl from February 1, 2027. Publishers relying on older NoArchive and NoCache signals for training or grounding must migrate by March 1, 2027 if they want to preserve those restrictions. Bing Webmaster Tools will offer a temporary transition mechanism for qualifying sites.

OpenAI says it intends to implement the common signal in the coming months. It has not committed to a precise activation date, and its existing crawler directives will continue to be respected.

For SEO and content teams, the decision should be documented before engineers touch robots.txt. Search indexing, foundation-model training and AI answer grounding are separate uses. Microsoft explicitly warns that restricting grounding can reduce visibility and citations in AI results without changing eligibility for conventional Bing search listings. Publishers should review those trade-offs, verify the behavior of each provider and avoid assuming that a draft token already controls every crawler.

Sources

Google Search Central — shared AI training control, 30 October 2026

IETF — Internet-Draft specification, 30 October 2026

Website operators may soon be able to express a single AI training preference to several major model providers. Google, Microsoft and OpenAI each announced support on October 30 for a shared signal in robots.txt, based on work at the IETF. This is a planned implementation of a draft approach, not an immediate universal rule: companies have different adoption schedules and existing crawler controls remain relevant.

Google says a shared policy token will let publishers communicate a broad training preference without maintaining a separate opt-out for every participating provider. If the token disallows training and the site has not explicitly configured Google-Extended, Google intends to interpret it as a Google-Extended disallow. Site-specific Google-Extended choices will still take precedence where specified. Google targets 2027 for support, and notes that sites explicitly allowing Google-Extended can remain eligible for Gemini Apps responses that include source links.

Microsoft's approach adds a distinction that matters for publishers: training a generative model is not the same operation as grounding an AI-generated answer in a live web page. Its Bing implementation supports training preferences in robots.txt and on individual pages, while grounding choices are page-level. The DisallowExtended option permits grounding within Microsoft-operated experiences but not through Microsoft grounding services supplied to third parties.

Microsoft says it will begin honoring the new preferences on recrawl from February 1, 2027. Publishers relying on older NoArchive and NoCache signals for training or grounding must migrate by March 1, 2027 if they want to preserve those restrictions. Bing Webmaster Tools will offer a temporary transition mechanism for qualifying sites.

OpenAI says it intends to implement the common signal in the coming months. It has not committed to a precise activation date, and its existing crawler directives will continue to be respected.

For SEO and content teams, the decision should be documented before engineers touch robots.txt. Search indexing, foundation-model training and AI answer grounding are separate uses. Microsoft explicitly warns that restricting grounding can reduce visibility and citations in AI results without changing eligibility for conventional Bing search listings. Publishers should review those trade-offs, verify the behavior of each provider and avoid assuming that a draft token already controls every crawler.

Sources

Google Search Central — shared AI training control, 30 October 2026

IETF — Internet-Draft specification, 30 October 2026

Website operators may soon be able to express a single AI training preference to several major model providers. Google, Microsoft and OpenAI each announced support on October 30 for a shared signal in robots.txt, based on work at the IETF. This is a planned implementation of a draft approach, not an immediate universal rule: companies have different adoption schedules and existing crawler controls remain relevant.

Google says a shared policy token will let publishers communicate a broad training preference without maintaining a separate opt-out for every participating provider. If the token disallows training and the site has not explicitly configured Google-Extended, Google intends to interpret it as a Google-Extended disallow. Site-specific Google-Extended choices will still take precedence where specified. Google targets 2027 for support, and notes that sites explicitly allowing Google-Extended can remain eligible for Gemini Apps responses that include source links.

Microsoft's approach adds a distinction that matters for publishers: training a generative model is not the same operation as grounding an AI-generated answer in a live web page. Its Bing implementation supports training preferences in robots.txt and on individual pages, while grounding choices are page-level. The DisallowExtended option permits grounding within Microsoft-operated experiences but not through Microsoft grounding services supplied to third parties.

Microsoft says it will begin honoring the new preferences on recrawl from February 1, 2027. Publishers relying on older NoArchive and NoCache signals for training or grounding must migrate by March 1, 2027 if they want to preserve those restrictions. Bing Webmaster Tools will offer a temporary transition mechanism for qualifying sites.

OpenAI says it intends to implement the common signal in the coming months. It has not committed to a precise activation date, and its existing crawler directives will continue to be respected.

For SEO and content teams, the decision should be documented before engineers touch robots.txt. Search indexing, foundation-model training and AI answer grounding are separate uses. Microsoft explicitly warns that restricting grounding can reduce visibility and citations in AI results without changing eligibility for conventional Bing search listings. Publishers should review those trade-offs, verify the behavior of each provider and avoid assuming that a draft token already controls every crawler.

Sources

Google Search Central — shared AI training control, 30 October 2026

IETF — Internet-Draft specification, 30 October 2026

Website operators may soon be able to express a single AI training preference to several major model providers. Google, Microsoft and OpenAI each announced support on October 30 for a shared signal in robots.txt, based on work at the IETF. This is a planned implementation of a draft approach, not an immediate universal rule: companies have different adoption schedules and existing crawler controls remain relevant.

Google says a shared policy token will let publishers communicate a broad training preference without maintaining a separate opt-out for every participating provider. If the token disallows training and the site has not explicitly configured Google-Extended, Google intends to interpret it as a Google-Extended disallow. Site-specific Google-Extended choices will still take precedence where specified. Google targets 2027 for support, and notes that sites explicitly allowing Google-Extended can remain eligible for Gemini Apps responses that include source links.

Microsoft's approach adds a distinction that matters for publishers: training a generative model is not the same operation as grounding an AI-generated answer in a live web page. Its Bing implementation supports training preferences in robots.txt and on individual pages, while grounding choices are page-level. The DisallowExtended option permits grounding within Microsoft-operated experiences but not through Microsoft grounding services supplied to third parties.

Microsoft says it will begin honoring the new preferences on recrawl from February 1, 2027. Publishers relying on older NoArchive and NoCache signals for training or grounding must migrate by March 1, 2027 if they want to preserve those restrictions. Bing Webmaster Tools will offer a temporary transition mechanism for qualifying sites.

OpenAI says it intends to implement the common signal in the coming months. It has not committed to a precise activation date, and its existing crawler directives will continue to be respected.

For SEO and content teams, the decision should be documented before engineers touch robots.txt. Search indexing, foundation-model training and AI answer grounding are separate uses. Microsoft explicitly warns that restricting grounding can reduce visibility and citations in AI results without changing eligibility for conventional Bing search listings. Publishers should review those trade-offs, verify the behavior of each provider and avoid assuming that a draft token already controls every crawler.

Sources

Google Search Central — shared AI training control, 30 October 2026

IETF — Internet-Draft specification, 30 October 2026