OpenAI has added more potential sources for ChatGPT Search through a joint research project with Cloudflare. The companies say network signals helped uncover and index hundreds of millions of previously unknown web pages. The published findings do not yet show whether that larger index has produced more citations or visits for publishers.

What the pilot has established

Announced on July 8, 2026, the pilot produced its first public results in a Cloudflare blog post on October 15. The participating group represented roughly 1% of Cloudflare’s overall network. The companies have not identified the participants or clarified whether the newly indexed pages belonged to their websites.

The approach helps crawlers locate useful information even when it attracts few inbound links. Examples include instructions for a particular appliance, current document application requirements and an up-to-date restaurant menu. Such pages can answer specific questions despite having a limited link profile.

How network signals guide crawling

Data from participating sites includes content freshness, traffic quality and evidence of actual page changes. These signals help OpenAI decide which URLs deserve a visit and when an existing page needs to be checked again.

They can also reveal requests that are unlikely to be useful. Automated traffic repeatedly targets missing files, duplicate content and URLs that simply redirect to a homepage. Identifying these patterns before fetching a page leaves more crawling resources available for useful material.

Another part of the research tests signals for potentially harmful pages. An attacker might, for instance, insert a fraudulent login form into an otherwise legitimate website. The aim is to flag that compromised page without treating the entire domain as unsafe.

Website owners retain access controls

A discovery signal does not authorize a crawler to retrieve a page. The companies describe the research as using web metadata for search and discovery, rather than supplying page content for model training.

Site owners can set separate rules for OAI-SearchBot, which supports search, and GPTBot, which collects material that may be used for training. Allowing the former while blocking the latter keeps a site available to ChatGPT Search while restricting training use of crawled content. Blocking OAI-SearchBot removes pages from search answers, although navigational links to the website may still appear.

Cloudflare’s AI Crawl Control, available on every plan, lets owners inspect AI crawler activity and allow or block individual crawlers. Server logs provide another way to examine those requests.

What a larger index means for publishers

An indexed page becomes a potential source for an answer. OpenAI still makes its own assessment and ranking decisions, however, so discovery through the pilot does not ensure a citation in ChatGPT.

The companies have not released citation rates, referral totals or measurements of reduced server load. Evaluating newly discovered pages can initially generate additional crawling. Referral measurement also becomes harder when a link lacks a UTM tag or does not pass referrer information.

What the next phase will measure

Further research will refine page selection, test quicker and more dependable change signals, and track what happens after content is discovered. The companies want to establish how improved discovery translates into business results for websites.

Cloudflare previously outlined plans to make the capability more widely available during 2026. The October 15 update did not provide a specific launch date.

OpenAI has added more potential sources for ChatGPT Search through a joint research project with Cloudflare. The companies say network signals helped uncover and index hundreds of millions of previously unknown web pages. The published findings do not yet show whether that larger index has produced more citations or visits for publishers.

What the pilot has established

Announced on July 8, 2026, the pilot produced its first public results in a Cloudflare blog post on October 15. The participating group represented roughly 1% of Cloudflare’s overall network. The companies have not identified the participants or clarified whether the newly indexed pages belonged to their websites.

The approach helps crawlers locate useful information even when it attracts few inbound links. Examples include instructions for a particular appliance, current document application requirements and an up-to-date restaurant menu. Such pages can answer specific questions despite having a limited link profile.

How network signals guide crawling

Data from participating sites includes content freshness, traffic quality and evidence of actual page changes. These signals help OpenAI decide which URLs deserve a visit and when an existing page needs to be checked again.

They can also reveal requests that are unlikely to be useful. Automated traffic repeatedly targets missing files, duplicate content and URLs that simply redirect to a homepage. Identifying these patterns before fetching a page leaves more crawling resources available for useful material.

Another part of the research tests signals for potentially harmful pages. An attacker might, for instance, insert a fraudulent login form into an otherwise legitimate website. The aim is to flag that compromised page without treating the entire domain as unsafe.

Website owners retain access controls

A discovery signal does not authorize a crawler to retrieve a page. The companies describe the research as using web metadata for search and discovery, rather than supplying page content for model training.

Site owners can set separate rules for OAI-SearchBot, which supports search, and GPTBot, which collects material that may be used for training. Allowing the former while blocking the latter keeps a site available to ChatGPT Search while restricting training use of crawled content. Blocking OAI-SearchBot removes pages from search answers, although navigational links to the website may still appear.

Cloudflare’s AI Crawl Control, available on every plan, lets owners inspect AI crawler activity and allow or block individual crawlers. Server logs provide another way to examine those requests.

What a larger index means for publishers

An indexed page becomes a potential source for an answer. OpenAI still makes its own assessment and ranking decisions, however, so discovery through the pilot does not ensure a citation in ChatGPT.

The companies have not released citation rates, referral totals or measurements of reduced server load. Evaluating newly discovered pages can initially generate additional crawling. Referral measurement also becomes harder when a link lacks a UTM tag or does not pass referrer information.

What the next phase will measure

Further research will refine page selection, test quicker and more dependable change signals, and track what happens after content is discovered. The companies want to establish how improved discovery translates into business results for websites.

Cloudflare previously outlined plans to make the capability more widely available during 2026. The October 15 update did not provide a specific launch date.

OpenAI has added more potential sources for ChatGPT Search through a joint research project with Cloudflare. The companies say network signals helped uncover and index hundreds of millions of previously unknown web pages. The published findings do not yet show whether that larger index has produced more citations or visits for publishers.

What the pilot has established

Announced on July 8, 2026, the pilot produced its first public results in a Cloudflare blog post on October 15. The participating group represented roughly 1% of Cloudflare’s overall network. The companies have not identified the participants or clarified whether the newly indexed pages belonged to their websites.

The approach helps crawlers locate useful information even when it attracts few inbound links. Examples include instructions for a particular appliance, current document application requirements and an up-to-date restaurant menu. Such pages can answer specific questions despite having a limited link profile.

How network signals guide crawling

Data from participating sites includes content freshness, traffic quality and evidence of actual page changes. These signals help OpenAI decide which URLs deserve a visit and when an existing page needs to be checked again.

They can also reveal requests that are unlikely to be useful. Automated traffic repeatedly targets missing files, duplicate content and URLs that simply redirect to a homepage. Identifying these patterns before fetching a page leaves more crawling resources available for useful material.

Another part of the research tests signals for potentially harmful pages. An attacker might, for instance, insert a fraudulent login form into an otherwise legitimate website. The aim is to flag that compromised page without treating the entire domain as unsafe.

Website owners retain access controls

A discovery signal does not authorize a crawler to retrieve a page. The companies describe the research as using web metadata for search and discovery, rather than supplying page content for model training.

Site owners can set separate rules for OAI-SearchBot, which supports search, and GPTBot, which collects material that may be used for training. Allowing the former while blocking the latter keeps a site available to ChatGPT Search while restricting training use of crawled content. Blocking OAI-SearchBot removes pages from search answers, although navigational links to the website may still appear.

Cloudflare’s AI Crawl Control, available on every plan, lets owners inspect AI crawler activity and allow or block individual crawlers. Server logs provide another way to examine those requests.

What a larger index means for publishers

An indexed page becomes a potential source for an answer. OpenAI still makes its own assessment and ranking decisions, however, so discovery through the pilot does not ensure a citation in ChatGPT.

The companies have not released citation rates, referral totals or measurements of reduced server load. Evaluating newly discovered pages can initially generate additional crawling. Referral measurement also becomes harder when a link lacks a UTM tag or does not pass referrer information.

What the next phase will measure

Further research will refine page selection, test quicker and more dependable change signals, and track what happens after content is discovered. The companies want to establish how improved discovery translates into business results for websites.

Cloudflare previously outlined plans to make the capability more widely available during 2026. The October 15 update did not provide a specific launch date.

OpenAI has added more potential sources for ChatGPT Search through a joint research project with Cloudflare. The companies say network signals helped uncover and index hundreds of millions of previously unknown web pages. The published findings do not yet show whether that larger index has produced more citations or visits for publishers.

What the pilot has established

Announced on July 8, 2026, the pilot produced its first public results in a Cloudflare blog post on October 15. The participating group represented roughly 1% of Cloudflare’s overall network. The companies have not identified the participants or clarified whether the newly indexed pages belonged to their websites.

The approach helps crawlers locate useful information even when it attracts few inbound links. Examples include instructions for a particular appliance, current document application requirements and an up-to-date restaurant menu. Such pages can answer specific questions despite having a limited link profile.

How network signals guide crawling

Data from participating sites includes content freshness, traffic quality and evidence of actual page changes. These signals help OpenAI decide which URLs deserve a visit and when an existing page needs to be checked again.

They can also reveal requests that are unlikely to be useful. Automated traffic repeatedly targets missing files, duplicate content and URLs that simply redirect to a homepage. Identifying these patterns before fetching a page leaves more crawling resources available for useful material.

Another part of the research tests signals for potentially harmful pages. An attacker might, for instance, insert a fraudulent login form into an otherwise legitimate website. The aim is to flag that compromised page without treating the entire domain as unsafe.

Website owners retain access controls

A discovery signal does not authorize a crawler to retrieve a page. The companies describe the research as using web metadata for search and discovery, rather than supplying page content for model training.

Site owners can set separate rules for OAI-SearchBot, which supports search, and GPTBot, which collects material that may be used for training. Allowing the former while blocking the latter keeps a site available to ChatGPT Search while restricting training use of crawled content. Blocking OAI-SearchBot removes pages from search answers, although navigational links to the website may still appear.

Cloudflare’s AI Crawl Control, available on every plan, lets owners inspect AI crawler activity and allow or block individual crawlers. Server logs provide another way to examine those requests.

What a larger index means for publishers

An indexed page becomes a potential source for an answer. OpenAI still makes its own assessment and ranking decisions, however, so discovery through the pilot does not ensure a citation in ChatGPT.

The companies have not released citation rates, referral totals or measurements of reduced server load. Evaluating newly discovered pages can initially generate additional crawling. Referral measurement also becomes harder when a link lacks a UTM tag or does not pass referrer information.

What the next phase will measure

Further research will refine page selection, test quicker and more dependable change signals, and track what happens after content is discovered. The companies want to establish how improved discovery translates into business results for websites.

Cloudflare previously outlined plans to make the capability more widely available during 2026. The October 15 update did not provide a specific launch date.