What is OAI-SearchBot, ChatGPT's search crawler?

OAI-SearchBot is the name OpenAI's search crawler gives your server. "OAI" is OpenAI, and "SearchBot" says what it is for: it does not train models or answer a specific person's question, it collects the pages that can appear in ChatGPT Search answers. For a store it is the OpenAI agent that decides whether you can show up in those answers, and it is easy to block by accident, tucked inside a list of "AI bots" copied from another site.

How it works

OpenAI describes its crawlers on a single page, and says this about OAI-SearchBot: it is used to surface websites in the results of ChatGPT's search features, and sites that opt out of it will not be shown in ChatGPT search answers, though they can still appear as navigational links.⁠^{[1]OpenAI. Overview of OpenAI Crawlers} It is an exclusion rule: the documentation says what happens if you block it, and does not promise that letting it in leads to an answer.

The exclusion happens in robots.txt, and OpenAI stresses that each agent is managed separately: you can allow OAI-SearchBot to appear in search results and block GPTBot to signal that crawled content should not be used to train its models. If you allow both, OpenAI may use a single crawl for both purposes. And for search, it estimates about 24 hours from the moment you change your robots.txt until its systems adjust.⁠^{[1]OpenAI. Overview of OpenAI Crawlers}

The request arrives with this user agent, in the version OpenAI documents as of October 2026:⁠^{[1]OpenAI. Overview of OpenAI Crawlers}

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot

The string starts like a desktop browser's, so a filter that looks only for "Mozilla" or "Chrome" will not tell it apart. What identifies it is OAI-SearchBot. In robots.txt that name is enough: the standard says the crawler looks for the group that names it, without regard to case, and if it finds none, follows the User-agent: * group.⁠^{[2]IETF (Koster, Illyes, Zeller and Sassman). RFC 9309: Robots Exclusion Protocol}

Because anyone can type that name into a request, OpenAI publishes the IP addresses it crawls from at openai.com/searchbot.json.⁠^{[1]OpenAI. Overview of OpenAI Crawlers} The file carries a creation date and a list of network prefixes: the version we read on October 6, 2026 was dated January 2, 2026 and held 40 IPv4 prefixes.⁠^{[3]OpenAI. searchbot.json (OAI-SearchBot IP ranges)} A visit whose IP is not on that list does not come from any of the addresses OpenAI declares for its search agent.

Crawling is not the only way your products reach ChatGPT. OpenAI also takes in structured product feeds through ACP, its agentic commerce protocol, so ChatGPT can index and show products with up-to-date price and availability.⁠^{[4]OpenAI. Agentic Commerce (product feeds)} On Shopify, blocking AI crawlers in robots.txt or at the network only affects what is discovered on the open web, and does not stop the product data Shopify Catalog sends to any agentic storefronts you have turned on.⁠^{[5]Shopify Help Center. Editing robots.txt.liquid}

In a store

A ceramics store copies a list from a forum to "protect itself from AI" and pastes it into its robots.txt. This is a fictional example:

User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ClaudeBot
Disallow: /

The intent was not to hand the catalog over to train models. The effect is different: with OAI-SearchBot in that group, the store disappears from ChatGPT's search answers as soon as OpenAI reads the change.⁠^{[1]OpenAI. Overview of OpenAI Crawlers} The version that does what was wanted takes OAI-SearchBot out of the group and leaves it under the general rules, which still protect the cart:

User-agent: *
Disallow: /cart
Disallow: /checkout

User-agent: GPTBot
User-agent: ClaudeBot
Disallow: /

Giving it its own group with Allow: / looks more explicit, but it has a cost: a crawler that finds its own group stops reading the asterisk group,⁠^{[2]IETF (Koster, Illyes, Zeller and Sassman). RFC 9309: Robots Exclusion Protocol} so it could also request the cart and checkout, and multiply faceted navigation URLs and duplicate content if the general group was blocking them. On BigCommerce there is another layer: the platform applies its own rules to AI and high-volume crawlers, with a 10-second delay, which you only see if you open the published file.⁠^{[6]BigCommerce Support. Understanding the Robots.txt File}

How to check it

1. Open yourdomain.com/robots.txt and look for a group that names OAI-SearchBot. If there is one, read what it blocks; if not, see what the asterisk group says about the path of your product pages.⁠^{[2]IETF (Koster, Illyes, Zeller and Sassman). RFC 9309: Robots Exclusion Protocol}
2. On BigCommerce, read the published file, not the settings screen, because the rules for AI bots only appear there.⁠^{[6]BigCommerce Support. Understanding the Robots.txt File}
3. Search your server or CDN logs for OAI-SearchBot and note which pages it requests.
4. Check each IP against openai.com/searchbot.json from the same day.⁠^{[3]OpenAI. searchbot.json (OAI-SearchBot IP ranges)}
5. If you changed something, wait at least a day before looking at ChatGPT: OpenAI estimates about 24 hours for its search to reflect the change.⁠^{[1]OpenAI. Overview of OpenAI Crawlers}

Related concept: ChatGPT-User

OAI-SearchBot crawls on its own so your pages can appear in ChatGPT search. ChatGPT-User does not crawl: it visits a page when a person asks a question in ChatGPT, and OpenAI warns that, since someone initiated the visit, robots.txt may not apply.⁠^{[1]OpenAI. Overview of OpenAI Crawlers} You will see both in your logs, and they are worth reading separately: one says your store is on search's radar; the other, that someone just asked about something you carry.

To keep going, the guide to what an AI crawler sees in your store goes agent by agent through what each one reads, and the reading path for this topic lays out the rest of the terms.