[The Engines]
How Can Brands Ensure Their Pages Are Found by ChatGPT's Expanded Search?
A Cloudflare and OpenAI pilot found hundreds of millions of previously unknown pages for ChatGPT Search. Brands need clear crawl rules, current pages, and proof that their content deserves a citation.
Explore this article with AI
Open a source-aware analysis with this article as the primary source.The short answer
ChatGPT search expansion gives OpenAI more paths to discover pages that ordinary link-following crawlers may miss. The October 15, 2026 Cloudflare and OpenAI pilot used network signals from about 1% of Cloudflare's network and enabled OpenAI to discover and index hundreds of millions of new pages. Brands should keep important pages accessible, current, distinct, and governed by deliberate AI crawler policies.
What changed in ChatGPT search expansion?
ChatGPT search expansion now has access to discovery signals that can surface pages beyond the traditional link graph. Cloudflare and OpenAI reported on October 15, 2026 that their research pilot enabled OpenAI to discover and index hundreds of millions of new web pages.
The pilot used a participating cohort of about 1% of Cloudflare's total network. Cloudflare says its network signals can help OpenAI's search crawler discover more useful pages, while OpenAI independently evaluates and ranks pages for relevance and usefulness.
That distinction matters for brands. Discovery gets a URL into the set of pages a search system can consider. It does not secure inclusion in an answer, a citation, a high placement, or a customer visit. A page still needs to give ChatGPT Search a clear reason to use it.
The expansion changes the starting point for many pages. A strong page that had little link equity or weak internal discovery paths may have a better chance of being found. A thin page remains thin after discovery. More crawl intelligence does not turn generic copy into a source worth citing.
- Before October 15, 2026, link-following discovery and known-URL recrawls were the familiar route into a crawler's view of a site.
- After the pilot results, Cloudflare network signals can help OpenAI identify previously unknown pages that may be useful to ChatGPT Search.
- OpenAI still evaluates and ranks discovered pages independently.
- Brands still control crawler access through their own technical policies and Cloudflare AI Crawl Control where they use Cloudflare.
Why can link-following crawlers miss useful brand pages?
Link-following crawlers can miss pages when few other pages point to them, even when those pages contain specific information people need. Cloudflare gives examples such as an exact appliance manual, current passport requirements, a local menu, and opening hours.
This is a practical problem for brands with deep documentation, local landing pages, product support content, compatibility pages, service-area information, or pages created for a narrow customer question. A page can serve a real audience and still sit outside the strongest routes a conventional crawler follows.
ChatGPT search expansion uses a different kind of signal. Cloudflare describes signals from its network that can help a crawler identify pages people reach, find pages that have changed, and avoid wasting crawl resources on broken addresses or duplicate routes.
The lesson is not that brands can ignore site architecture. Internal links, sitemaps, canonical tags, and clean navigation remain useful ways to make a site understandable. The new discovery route gives brands another reason to publish precise pages that answer real questions instead of relying on a small set of broad marketing URLs.
| Period | Discovery condition | What Cloudflare and OpenAI reported | What brands should do |
|---|---|---|---|
| Before October 15, 2026 | Conventional link-following discovery can leave low-linked pages outside a crawler's view. | Cloudflare describes useful pages that may have few links pointing to them. | Keep priority pages linked internally, included in sitemaps, and free of technical errors. |
| October 15, 2026 pilot result | Cloudflare network signals can help OpenAI find pages beyond the existing link graph. | A pilot using about 1% of Cloudflare's network enabled OpenAI to discover and index hundreds of millions of new pages. | Audit specific pages that answer customer questions, especially deep documentation and narrow service information. |
| Next pilot phase | Discovery alone does not establish a business outcome. | Cloudflare says the next phase will measure what happens after content is discovered. | Track Citation Share and Answer Presence alongside referrals, leads, bookings, or signups. |
Who does ChatGPT search expansion affect first?
ChatGPT search expansion affects brands whose best information lives on pages with limited inbound links or uneven internal visibility. That includes companies with detailed help centers, model-specific documentation, location pages, complex product catalogs, and frequently updated operational information.
It also affects teams that have treated a page as complete once it is published. A page may now be easier for ChatGPT Search to find, but discovery exposes weaknesses as well as strengths. If the page is outdated, duplicates another URL, redirects incorrectly, or fails to answer the question in its title, expanded discovery does not create a better outcome.
The strongest candidates are pages with a direct purpose. They identify the topic, give a specific answer, show who the answer applies to, and retain enough context for a user or system to verify the point. That is the basic standard for content that can earn a citation.
For Omnicite's audience, the implication is simple. Search visibility still matters, but citation visibility becomes the sharper question. A brand should ask whether its pages can be found, understood, trusted, and selected when a user asks ChatGPT for a category recommendation, comparison, local provider, or practical answer.
How does the pilot use Cloudflare signals?
The pilot uses metadata about the web for search and discovery. Cloudflare says that metadata is used instead of sharing page content for model training, and that a signal about a page does not grant a crawler permission to access it.
Cloudflare describes change signals that help a crawler decide when a known page merits another visit. It also describes filtering that can keep crawlers away from nonexistent pages, redirects, and duplicate content. Those checks can direct crawl resources toward pages more likely to help users.
The pilot also tests signals for potentially harmful pages. Cloudflare says a legitimate site can contain one compromised URL, such as a fake login page inserted by an attacker. The stated goal is to identify pages that should stay out of an index without treating the entire domain as unsafe.
Brands should read this as an operational message. Technical hygiene now influences more than classic search crawling. Broken routes, uncontrolled parameter pages, redirect-all behavior, and neglected security problems can make it harder for a discovery system to identify the pages that deserve attention.
What should brands do about ChatGPT search expansion now?
Brands should audit the pages that contain their most specific, decision-making information and make those pages easy to access, current, and distinct. Start with documentation, service pages, comparison pages, product specifications, location pages, and answers to recurring customer questions.
First, check that each priority page returns the right page instead of a generic redirect or an error. Cloudflare identifies nonexistent pages, duplicate content, and redirect behavior as sources of wasted crawling. A site that sends every bad URL to its homepage makes it harder to separate a legitimate page from noise.
Second, give every priority page a direct answer near the top. ChatGPT Search needs source material that resolves a question. State what the product, service, policy, or location offers. Define limits and update dates where they change. Support important claims with evidence a reader can inspect.
Third, review crawler permissions deliberately. Cloudflare says AI Crawl Control gives site owners visibility into AI services accessing their content and tools to manage access according to their preferences. Access settings should reflect the brand's commercial, legal, and editorial position, not an accidental default.
Fourth, measure citations separately from traffic. Better discovery can increase the pool of pages ChatGPT may evaluate, yet the pilot's next phase is still focused on understanding what happens after content is discovered. Track Citation Share, Citation Count per day, Answer Presence, and Share of Voice so the team can see whether discoverability becomes meaningful brand presence.
- Identify the pages that answer high-intent customer questions.
- Fix broken routes, duplicate pages, and catch-all redirects.
- Refresh facts, availability details, and supporting evidence before those pages are rediscovered.
- Set and document crawler permissions for the site.
- Monitor citation outcomes across the AI engines that matter to the business.
Does expanded discovery guarantee a ChatGPT citation?
Expanded discovery does not guarantee a ChatGPT citation. Cloudflare says OpenAI independently evaluates and ranks discovered pages based on relevance and helpfulness, so a discovered URL enters a selection process rather than receiving a promised outcome.
That is why content quality remains the work. A page needs a focused subject, accurate details, a visible update path, and evidence appropriate to the claim. Pages that repeat vague category language give a system little reason to cite them over a primary source, specialist guide, or competitor with clearer coverage.
A citation also depends on the question. One page may be highly useful for a troubleshooting prompt and irrelevant for a buying comparison. Brands need topic coverage that maps to the actual questions customers ask instead of treating one broad page as an answer to every intent.
Omnicite calls this Citation Engineering. The work is to build authoritative content with enough coverage, quality, and freshness for AI systems to trust, then measure whether that work creates citation share. It is not an attempt to game or manipulate models.
How should teams balance access control with discoverability?
Teams should make an explicit access decision for each important content area. Cloudflare states that domain owners' choices matter throughout the pilot and points site owners to AI Crawl Control to understand and manage how AI systems access their websites.
The practical decision has four parts. Identify which content supports discovery, separate public pages from restricted material, establish who owns crawler-policy changes, and record why a policy exists. That record prevents a marketing objective from silently overriding a security or legal requirement.
Access control and discoverability can coexist when the policy is intentional. A brand can expose public documentation that helps customers while protecting account pages, internal tools, gated resources, and sensitive routes. The correct boundary depends on the business and its obligations.
The October 2026 pilot gives brands a reason to revisit that boundary now. If a company wants ChatGPT Search to find its public expertise, its accessible pages need to be worth finding. If it does not want a crawler to access a route, that restriction should be clear and tested.
What should leaders measure after the expansion?
Leaders should measure whether expanded discovery leads to credible answer presence and citations for their priority questions. Cloudflare says the next phase of the pilot will improve page selection, test faster change signals, and measure what happens after content is discovered.
That means traffic alone is an incomplete scorecard. A brand can be cited in a ChatGPT answer before a user visits its website, and a visit can occur without a citation. The useful measurement system separates visibility in answers from downstream business performance.
Use Citation Share as the headline measure: the percentage of relevant AI answers in a category that cite the brand. Pair it with Answer Presence to show breadth across the question universe, Citation Count per day to show volume, and Share of Voice to compare named competitors.
Then connect the data to pages and prompts. If a brand has no citations for a high-intent question, inspect whether it has a direct page for that question, whether the page remains current, and whether competing sources provide clearer evidence. That is a concrete editorial backlog, not a guess about an opaque ranking system.
Key takeaways
- ChatGPT search expansion can surface pages that link-following crawlers may miss.
- Cloudflare and OpenAI reported hundreds of millions of newly discovered and indexed pages from a pilot using about 1% of Cloudflare's network.
- Discovery creates an opportunity for evaluation, not a guaranteed citation or visit.
- Brands should prioritize current, specific pages that directly answer important customer questions.
- Crawler access needs a deliberate policy, especially for public expertise, restricted routes, and sensitive content.
- Citation Share and Answer Presence show whether expanded discovery creates real AI search visibility.
Omnicite Editorial. "ChatGPT Search Expansion: What Brands Should Do" The Citation Report, Omnicite. https://omnicite.co/blog/how-can-brands-ensure-their-pages-are-found-by-c/
Sources
Source: Cloudflare
Cloudflare and OpenAI reported that a research pilot using about 1% of Cloudflare's network enabled OpenAI to discover and index hundreds of millions of new web pages. The source also states that OpenAI independently evaluates and ranks pages. Cloudflare, 2026-10-15
Source: Cloudflare Developers
Cloudflare AI Crawl Control provides visibility into AI services accessing site content and tools to manage that access according to site-owner preferences. Cloudflare Developers, 2026-08-14
Frequently asked questions
What is ChatGPT search expansion?
ChatGPT search expansion describes the broader discovery capability reported in the Cloudflare and OpenAI research pilot. Cloudflare says network signals helped OpenAI discover and index hundreds of millions of previously unknown web pages for ChatGPT Search.
Does ChatGPT search expansion mean every page can appear in ChatGPT?
No. Discovery does not guarantee inclusion in an answer or a citation. Cloudflare says OpenAI independently evaluates and ranks discovered pages based on relevance and helpfulness.
What kinds of pages may benefit from expanded discovery?
Pages with narrow, useful information and limited links may benefit, such as product documentation, local menus, service details, current requirements, model-specific guides, and other pages that answer a specific question.
Does Cloudflare share page content with OpenAI for model training in this pilot?
Cloudflare says the research uses metadata about the web for search and discovery. Its October 15, 2026 post states that page content is not shared for model training through this arrangement.
Can a site owner control AI crawler access?
Yes. Cloudflare states that AI Crawl Control gives site owners visibility into AI services accessing their content and tools to manage access according to their preferences. A discovery signal does not itself grant permission to access a page.
What should a brand measure after ChatGPT search expansion?
Measure whether the brand appears and is cited for priority questions. Citation Share, Answer Presence, Citation Count per day, and Share of Voice show different parts of AI search visibility, while business metrics show whether that visibility creates a commercial result.