AI Search Optimization Guide (2026): How to Get Cited by ChatGPT, Perplexity and AI Overviews
AI assistants answer questions by composing text and citing a few sources. Being one of those sources depends on three things you control: whether their crawlers can read your pages, whether your content states answers plainly enough to quote, and whether machines can tell who you are.
- Most AI crawlers do not execute JavaScript, so client-rendered content is invisible to them.
- Allowing the crawlers in robots.txt is necessary but not sufficient; firewalls and bot protection often block them anyway.
- Assistants cite specific facts, definitions and original data, not generic overviews.
- Nobody can guarantee a citation. Measure referrals and improve the known factors.
How AI search works, briefly
Assistants such as ChatGPT, Perplexity, Gemini and Copilot, and Google's AI Overviews, retrieve documents, then generate an answer grounded in them. Some maintain their own index built by their own crawler; others rely on a search engine's index. Either way, a page that cannot be fetched and read in plain HTML is not a candidate.
Step one: let the crawlers in
Check robots.txt for rules that block AI user agents. The common ones are GPTBot and OAI-SearchBot (OpenAI), PerplexityBot, ClaudeBot (Anthropic), Google-Extended (Google's AI training control, separate from Googlebot), and Applebot-Extended. Then check the layer people forget: a CDN or firewall bot-protection rule can block these agents even when robots.txt allows them.
- Decide per crawler. Blocking training crawlers and allowing search crawlers is a legitimate choice.
- Test with the crawler's user agent and confirm a 200 response with full content.
- Keep your sitemap current; several AI crawlers use it.
Step two: make sure the content is in the HTML
Most AI crawlers fetch the HTML and do not run scripts. If your content is rendered in the browser by JavaScript, what they receive is an empty shell. Server-side rendering, static generation, or a server-injected content block fixes this. View the page source, not the inspector: if the text is not in the source, assume AI crawlers cannot see it.
Step three: write so the answer can be quoted
- State the answer in the first paragraph, in a complete sentence that makes sense out of context.
- Use headings that are the questions people ask.
- Give specific numbers, dates and definitions, with sources.
- Keep one topic per page so the page is an unambiguous match.
- Add a short summary or key takeaways block.
Step four: be a recognisable entity
Assistants prefer sources they can attribute. Use Organization markup with a consistent name, logo and profiles; author markup with real people and credentials; and keep your name, address and descriptions consistent across the web. An llms.txt file listing your most useful pages is a cheap addition.
Step five: measure
Track referrals from chatgpt.com, perplexity.ai, gemini.google.com and copilot.microsoft.com in analytics, and watch server logs for AI crawler activity. Citations are harder to measure directly; periodic manual checks of the questions you care about are still the most reliable method.
What does not work
- Stuffing pages with questions to look answer-shaped.
- Hidden text or markup aimed at models.
- Mass-producing thin pages in the hope one is cited.
- Paying for tools that claim to guarantee placement.
Questions
Do AI crawlers respect robots.txt?
The major ones state that they do. Compliance varies among smaller crawlers.
Does blocking Google-Extended affect Google Search rankings?
Google says it does not. It controls use of content for AI training and grounding in some products, separately from Googlebot.
Is llms.txt an official standard?
No. It is a proposed convention that some tools read. It costs little and does no harm.
How is this different from SEO?
It overlaps heavily. The main differences are the emphasis on server-rendered HTML, quotable answers and entity clarity.