How ChatGPT search chooses the websites it cites
ChatGPT search turns a question into several searches, sends them to search providers such as Bing, reads the pages that come back and cites the ones it used. A site becomes eligible by being open to its crawler and indexed in ordinary search, and is chosen by being the clearest answer to the search it made.
Each sounding opens its section
- 01 What happens when someone asks 97 words
- 02 The crawler: OAI-SearchBot 59 words
- 03 The search providers 59 words
- 04 What makes a page quotable 72 words
- 05 Three crawlers, three jobs 67 words
- 06 How to tell whether it is working 64 words
- 07 Questions people ask 118 words
- 08 Free competitor gap report 101 words
- 09 More in this series 77 words
- Words
- 714
- Site 366–1,117 · median 749.5
- Sections (H2)
- 9
- Site 3–14 · median 9
- Internal links
- 16
- Site 1–36 · median 19
- File
- 71.4 kB
- HTML, uncompressed · of 83.6 kB budget · 0 scripts
What happens when someone asks
A question to ChatGPT does not always trigger a search. When it does, OpenAI describes three steps: the question is rewritten into one or more targeted searches, those are sent to search providers, and after reading the first results ChatGPT may send further, more specific searches. The answer is written from the pages it read, with links to them.
The searches it runs are often longer and more specific than what the person typed. A page written for one exact question has a better chance than a page that covers everything loosely.
The crawler: OAI-SearchBot
OpenAI runs a crawler named OAI-SearchBot to find pages for ChatGPT search. Two things make a site eligible, by OpenAI’s account: the crawler is allowed in robots.txt, and the site’s host or content delivery network lets its traffic through.
OpenAI publishes the addresses the crawler comes from, so a host can allow them by name.
The search providers
ChatGPT search also uses outside search providers, and OpenAI names Bing. That makes Bing’s index worth the same care as Google’s.
- Verify the site in Bing Webmaster Tools and submit the sitemap.
- Fix what Bing reports as crawl errors.
- Use IndexNow, where your platform supports it, to tell Bing about new and changed pages at once.
What makes a page quotable
An assistant quotes the page that makes its job easy.
- The answer is in the first paragraph, in a sentence that stands on its own.
- The page covers one subject, named in its title and main heading.
- Facts such as prices, areas served and opening hours are written as text.
- The page shows when it was last updated.
- Headings are phrased the way the questions are asked.
Three crawlers, three jobs
OpenAI runs more than one crawler, and they should not be confused. OAI-SearchBot finds pages for search answers. GPTBot collects content that may be used to train models. ChatGPT-User fetches a page when a person’s request needs it. Blocking the first removes a site from ChatGPT search; blocking the second does not. Our guide to whether to block AI crawlers covers the choice.
How to tell whether it is working
Two records show the effect. Google Analytics shows the visits an assistant sent, in a channel of their own: our guide to seeing ChatGPT traffic in Google Analytics explains the reports. The site’s server log shows the crawler itself arriving, which Analytics never records. The fuller picture is in how to get ChatGPT to recommend your business.
Questions people ask
Does ChatGPT use Google or Bing?
OpenAI says ChatGPT search uses third-party search providers and content from its partners, and it names Microsoft's Bing among them. It also runs its own crawler, OAI-SearchBot.
How do I rank in ChatGPT search?
Allow OAI-SearchBot to read the site, make sure the pages are indexed in ordinary search engines, and put a direct answer near the top of each page. ChatGPT cites pages that answer the specific search it ran.
Can I stop ChatGPT search from showing my site?
Yes. OpenAI states that a site which disallows OAI-SearchBot in robots.txt is not shown in ChatGPT search answers. That is separate from GPTBot, which collects content for training.
Free competitor gap report
A page is cited when it answers the search better than the alternatives. Name up to three competitors and we measure your pages against theirs on the same checks, then send you where theirs give a clearer answer.
- Speed and technical health, side by side
- On-page SEO and site structure
- The content they are found for and you are not
- Who holds the searches that matter in your market
- A prioritised list of what to fix first
Request your free report by email
- Address
- contact@hadalmedusa.com
- Include
- your site; up to three competitors; where you compete
More in this series
- How to check if ChatGPT recommends your business
- How to get ChatGPT to recommend your business
- What llms.txt is, with an example
- Should you block AI crawlers?
- How to see ChatGPT traffic in Google Analytics
- AI Overviews and falling clicks: how to check, and what to do
- Generative engine optimisation: what it means and what works
- AI search and answer engines: the whole topic
- All guides
- On-site SEO: what the service includes