How to Ensure Your Site is Cited by ChatGPT and Perplexity

What changed in AI Citation Optimization?

AI Citation Optimization now has a technical prerequisite that many teams miss: the crawler connected to answer visibility must be able to reach the site. Treating every AI-related crawler as one thing can block a site from search answers while leaving the team convinced it merely opted out of model training.

OpenAI documents separate controls for OAI-SearchBot and GPTBot. OAI-SearchBot is used to surface websites in ChatGPT search features, while GPTBot crawls content that may be used to train OpenAI foundation models. OpenAI states that a site can allow OAI-SearchBot and disallow GPTBot, and that the settings are independent.

Perplexity makes a similar distinction. Its documentation says PerplexityBot surfaces and links websites in Perplexity search results and is not used to crawl content for AI foundation models. The operational change is simple: stop making one blanket decision about AI crawling. Set an explicit policy for answer visibility and a separate policy for model training.

1. Review robots.txt before changing the content strategy.
2. Check web application firewall rules alongside robots.txt.
3. Verify that the hosting stack does not block known crawler traffic.
4. Document the owner of future crawler-policy changes.

Who does this affect most?

This affects any company that expects buyers to ask ChatGPT or Perplexity for recommendations, comparisons, definitions, implementation help, or local service options. It matters most when the company has strong subject knowledge but cannot tell whether an answer engine can access the pages where that knowledge lives.

B2B SaaS teams are exposed when category and comparison prompts shape shortlists. A product page can be excellent for a human visitor yet remain absent from an answer if the relevant crawler is blocked, the page cannot render, or the explanation is scattered across vague sections. The consequence is more than lost traffic. It is lost consideration before a buyer reaches a results page.

Local and multi-location businesses face the same issue through location-led prompts. A searcher who asks for a service in a city may receive a compact answer with only a few cited options. There is no page two in an AI answer, so citation readiness matters before a buyer makes the request.

1. Review sites that sit behind aggressive bot protection.
2. Review teams that previously blocked every AI user agent.
3. Review brands that rely on JavaScript-heavy templates.
4. Review companies that publish comparison or buyer-guide content.

What does the before-and-after crawler change look like?

The before-and-after is a change in operating model, not a promise that a robots.txt edit produces citations. Before the review, a site may use a broad block or make a single policy decision about AI bots. After the review, the site permits the crawlers used for answer visibility, applies training restrictions deliberately, and confirms that security controls allow legitimate requests.

OpenAI says its systems can take about 24 hours to adjust after a robots.txt update. Perplexity likewise says changes may take up to 24 hours to be reflected. That timing describes crawler-policy propagation. It does not establish that a page will be cited after a day, because citation depends on the engine response and the page's fit for the prompt.

Use the table as a dated implementation check. It separates the documented control from the content work that follows. A crawlable page still needs a direct answer, supporting evidence, clear scope, visible maintenance, and enough coverage of the prompts that define your category.

1. Take a dated copy of the current robots.txt file.
2. Record the relevant WAF rule and the current response status.
3. Deploy a narrowly scoped policy change.
4. Recheck access after the vendor-stated adjustment window.
5. Monitor Answer Presence and Citation Share on the same prompt set.

How should you configure ChatGPT access?

For ChatGPT search, allow OAI-SearchBot where you want pages eligible to surface in ChatGPT search features. OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, although they can still appear as navigational links. That makes OAI-SearchBot the key technical control for ChatGPT answer visibility.

Decide separately whether to allow GPTBot. OpenAI describes GPTBot as the crawler used for content that may be used in training its generative AI foundation models. A team that wants to restrict training use can disallow GPTBot while keeping OAI-SearchBot allowed. Do not let an old GPTBot decision silently become a decision about search visibility.

Validate the request path, not only the robots.txt syntax. OpenAI recommends allowing OAI-SearchBot in robots.txt and allowing requests from its published IP ranges. If a firewall, CDN rule, origin policy, or bot-management product rejects those requests, the intended permission may not work in practice.

1. Allow OAI-SearchBot for pages intended to appear in ChatGPT search.
2. Choose a GPTBot policy based on training preferences.
3. Use published IP ranges when configuring security controls.
4. Limit crawl access only where a real business or security need requires it.

How should you configure Perplexity access?

For Perplexity, allow PerplexityBot where you want pages eligible to surface and link in Perplexity search results. Perplexity states that PerplexityBot is not used to crawl content for AI foundation models. That removes the false trade-off between Perplexity answer visibility and training permission for this specific crawler.

Perplexity-User is different. Perplexity says it supports user actions and may visit a page to help answer a user request, while generally ignoring robots.txt rules because the user initiated the fetch. It is not used for web crawling or for collecting content for foundation-model training. Do not confuse a user-triggered fetch with the automatic crawler that supports broader discovery.

A secure implementation checks both the user agent and the published IP information. Perplexity explicitly provides guidance for web application firewalls and recommends combining user-agent and IP-address conditions in WAF rules. That is more defensible than trusting a header alone.

1. Allow PerplexityBot for citation-ready content.
2. Review WAF rules for PerplexityBot access.
3. Use Perplexity's published IP information for verification.
4. Do not use Perplexity-User behavior as a substitute for PerplexityBot access.

What content is most likely to earn a citation after access is fixed?

The best citation candidate is a page that answers one important prompt without forcing the engine to assemble the answer from marketing copy. Put the direct answer first, define the scope, explain the conditions that change the answer, and support material claims with primary or authoritative sources. A citation is easier to justify when the useful passage is complete on its own.

Build coverage around question clusters, rather than an undifferentiated pile of articles. A B2B SaaS company might need a category definition, an implementation guide, a buyer comparison, a pricing explainer, a security explanation, and a current integration page. A local business may need service pages that establish geography, service scope, credentials, constraints, and what happens next.

Freshness is part of the job because an old page can give an engine a reason to choose a newer source. Show publication or update dates when they are meaningful, update claims when the underlying product or market changes, and remove statements that no longer have evidence. Citation Engineering is disciplined publishing, not an attempt to game a model.

1. Answer the page's main question in the opening paragraph.
2. Use descriptive headings that mirror buyer questions.
3. Cite sources beside material facts.
4. Separate facts, opinions, and product claims.
5. Update pages when their evidence or operating details change.

How should you measure whether the response worked?

Measure the response with a fixed prompt set and distinguish visibility from performance. Citation Share is the percentage of relevant AI answers in a category that cite you. Citation Count per day measures volume. Answer Presence measures how broadly you appear across the question universe, while Share of Voice shows your position relative to named competitors.

Start with a baseline before changing crawler rules or rebuilding pages. Record the exact prompts, the engine, the date, the cited URLs, brand mentions, and competitor citations. Without a consistent prompt set, a later improvement can be mistaken for normal answer variation or a change in the request itself.

Review results by page and by request type. If a page becomes reachable but never appears, inspect whether it directly answers the prompt, whether its evidence is current, and whether a stronger page already owns that answer. If your brand appears without your domain being cited, the engine may know the brand but trust another source to explain it.

1. Set a repeatable category and comparison prompt set.
2. Capture cited domains and URLs for every answer.
3. Track Citation Share separately from Answer Presence.
4. Compare against named competitors on the same questions.
5. Prioritize gaps where a high-intent request has no reliable owned source.

What should your team do next?

Start with a crawler-access audit, then move to content coverage. The audit should identify whether OAI-SearchBot and PerplexityBot can access the templates that matter, whether training preferences are intentionally configured, and whether security tooling contradicts robots.txt. Fix binary access failures before debating schema, wording, or publishing volume.

Next, choose a small set of commercially important questions and audit the page that should answer each one. The aim is not to manufacture citations. The aim is to publish the most useful, well-supported answer your company can honestly provide, then make sure answer engines can fetch it.

Establish a recurring measurement routine. Track the same prompts across ChatGPT and Perplexity, inspect the citations rather than only the mentions, and feed the findings into the next editorial cycle. Rankings got you found. Citations get you chosen.

1. Audit crawler and WAF access.
2. Select the highest-intent question set.
3. Improve owned pages that should answer those questions.
4. Track Citation Share and Answer Presence over time.
5. Repeat the technical audit when infrastructure policies change.

Sources

Source: OpenAI

OAI-SearchBot surfaces websites in ChatGPT search features, while GPTBot crawls content that may be used for foundation-model training. OpenAI says the controls are independent and search adjustments can take about 24 hours after robots.txt changes. OpenAI, 2026-09-28

Source: Perplexity

PerplexityBot surfaces and links websites in Perplexity search results and is not used for AI foundation-model training. Perplexity says crawler-setting changes may take up to 24 hours to reflect. Perplexity, 2026-09-28

Source: Google for Developers

Google distinguishes automatic crawlers from user-triggered fetchers and states that common crawlers such as Googlebot respect robots.txt rules for automatic crawls. Google for Developers, 2026-06-12

Source: Pepper Content

The brief that prompted this news reaction identifies distinct crawler access as the practical change for ChatGPT and Perplexity optimization. Pepper Content, 2026-09-10