AI Search & GEO

What Is OAI-SearchBot?

By BacklinkPrice Team · Nov 18, 2026 · Updated Nov 20, 2026 · 12 min read
Illustrated cover for What Is OAI-SearchBot?

OAI-SearchBot is OpenAI's crawler for ChatGPT search. It fetches pages so ChatGPT can show them in search answers, and blocking it removes your pages from those answers.

OAI-SearchBot is OpenAI's crawler for ChatGPT search. It visits public web pages so ChatGPT can find them, show them in search answers, and link to them as sources, and blocking it removes your pages from those answers while leaving model training rules untouched.

Key takeaways

  • OAI-SearchBot is OpenAI's search crawler. It decides whether your pages can appear in ChatGPT search answers, and it is separate from GPTBot, the training crawler.
  • Sites that block OAI-SearchBot in robots.txt will not be shown in ChatGPT search answers, though they can still appear as plain navigational links.
  • OpenAI lists 4 user agents, and each robots.txt setting works on its own, so you can allow search crawling while refusing training use.
  • A robots.txt change can take about 24 hours to reach ChatGPT search, and a firewall that blocks OpenAI's published IP ranges can undo an allow rule.
  • Allowing the crawler makes a page eligible for ChatGPT search. It does not guarantee a citation, because OpenAI says placement is not guaranteed.

What OAI-SearchBot is

OAI-SearchBot is OpenAI's automated crawler for search. OpenAI's crawler documentation states that it surfaces websites in ChatGPT search features, which means it fetches pages so ChatGPT can show them and link to them when a question needs current web information.

The crawler is one of 4 agents OpenAI lists on its crawler page as of November 2026. The other 3 are GPTBot, which gathers content that may train models, ChatGPT-User, which visits a page when a person asks ChatGPT to read it, and OAI-AdsBot, which checks ad landing pages.

OAI-SearchBot runs on OpenAI's schedule, not when a user asks a question. That automatic crawl builds the pool of pages ChatGPT search can draw on, while ChatGPT-User handles live, one-off visits. Our guide to whether AI crawlers follow links explains how that automatic crawl treats links on your pages.

How OAI-SearchBot works

OAI-SearchBot works by fetching pages it is allowed to visit, reading the HTML it receives, and passing what it finds to ChatGPT search. OpenAI does not publish a crawl schedule, a crawl budget, or a list of ranking factors, so site owners control access, not the outcome.

The bot identifies itself with a long Chrome-style user agent string that ends in OAI-SearchBot/1.4, and OpenAI notes the version number may change.

When it fetches robots.txt itself, it may add a robots.txt marker to that string, which helps owners tell robots.txt requests apart from page requests in server logs. The full string is published in OpenAI's crawler documentation.

OpenAI publishes the bot's IP addresses as a JSON file at openai.com/searchbot.json.

The file opened on 19 November 2026 lists 39 IPv4 prefixes and a creation time of 2 January 2026, so you can check a request IP against that list before trusting a user agent string, which anyone can copy.

What OAI-SearchBot can read

The bot reads the HTML your server sends. Vercel and MERJ reported in December 2024 that none of the major AI crawlers they studied rendered JavaScript, and they named OAI-SearchBot, ChatGPT-User, and GPTBot in that finding.

That study is 2 years old, and OpenAI does not say whether its crawlers run scripts, so treat server-sent HTML as the baseline.

In the same December 2024 study, JavaScript files made up 11.50% of ChatGPT's crawler requests and 23.84% of Claude's, yet the crawlers downloaded those files without running them. That finding is quoted verbatim from the Vercel and MERJ study in 99Visibility's JavaScript SEO guide.

Data in the first HTML response can still reach the bot.

The Vercel and MERJ study noted that content in the initial HTML, such as JSON data, may be indexed, since AI models can interpret non-HTML content. A page that ships its facts inside the HTML beats an empty shell that fetches everything after load.

OAI-SearchBot vs GPTBot and ChatGPT-User

The 3 main agents differ by job, not by company. OAI-SearchBot feeds ChatGPT search, GPTBot gathers content that may train OpenAI's models, and ChatGPT-User opens a page because a person asked for it.

OpenAI's crawler documentation says each robots.txt setting is independent of the others, so a rule for one agent has no effect on the others.

AgentJobWhat blocking it does
OAI-SearchBotSurfaces sites in ChatGPT searchPages are not shown in ChatGPT search answers
GPTBotGathers content that may train modelsContent should not be used for training
ChatGPT-UserVisits a page on a user's requestRobots.txt rules may not apply
OAI-AdsBotChecks ChatGPT ad landing pagesAds cannot be checked, and only ad pages are visited

If you allow both OAI-SearchBot and GPTBot, OpenAI says it may use one crawl for both uses to avoid crawling twice. That is the only overlap OpenAI documents. A site that wants search visibility without training use should allow the first agent and disallow the second, as a separate rule for each.

Other vendors split the same jobs across their own agents. Perplexity runs PerplexityBot for its search index, and our guide to how Perplexity chooses sources shows how a second search index can cite pages ChatGPT misses.

OAI-SearchBot matters because it is the gate to ChatGPT search answers. OpenAI's help page says a site must allow the bot and let its published IP addresses through to be eligible, and it adds that ChatGPT ranks results with multiple factors and that "placement is not guaranteed."

OpenAI states the cost of getting this wrong in plain terms: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links."

That sentence is the whole reason to check your robots.txt file before you spend time on any other AI search work.

Web search in ChatGPT is available on 7 plans as of November 2026: Free, Go, Plus, Pro, Business, Enterprise, and Edu, and people who are not signed in can also use it.

That spread across free and paid plans means the crawler's allow rule is worth checking even for sites that get little traffic from Google today.

How big AI crawling got in 2025

The scale of AI crawling grew fast in 2025. Cloudflare reported in August 2025 that AI and search crawling rose 32% year over year in April 2025, stayed 24% higher in June, and slowed to 4% higher in July.

GPTBot's share of all AI and search crawler traffic rose from 4.7% in July 2024 to 11.7% in July 2025, while OAI-SearchBot rose from 0 to 0.9% over the same period. In AI-only traffic, OAI-SearchBot reached 2.2% in July 2025, and GPTBot reached 28.1%.

Training still takes most of the crawl volume. Cloudflare reported in August 2025 that training drove about 80% of AI bot activity over the prior 12 months, against 18% for search and 2% for user actions. In July 2025 alone, training was 79%, search was 17%, and user actions were 3.2%.

Crawling does not pay back in clicks at the same rate for every platform. Cloudflare reported in August 2025 that OpenAI crawled 1,091 pages for each referred visit in July 2025, against 5.4 for Google, 194.8 for Perplexity, and 38,065.7 for Anthropic.

For what happens after the crawl, see how ChatGPT chooses which sources to cite.

How to allow or block OAI-SearchBot

You control OAI-SearchBot with 2 files and settings: robots.txt and your host or firewall. OpenAI's help page states both as conditions for eligibility, so an allow rule in robots.txt alone is not enough if your network layer blocks the bot's IP addresses.

Step 1: Check your current robots.txt

Open yoursite.com/robots.txt in a browser and search for OAI-SearchBot. If no group names it, the User-agent: * group applies to it, so read that group too. Note any Disallow lines that cover the pages you want cited.

Step 2: Write the rule you want

To allow the bot on the whole site, add a group with User-agent: OAI-SearchBot and Allow: /. To block it from the whole site, use the same group with Disallow: /. To block training while staying in search, add a second group with User-agent: GPTBot and Disallow: /.

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

How to confirm OAI-SearchBot can reach your site

Allowing the bot in robots.txt is only half the job. These checks confirm that the bot can actually reach your pages.

Check the firewall and CDN

Open your firewall, CDN, or bot-protection settings and confirm that requests from OpenAI's published searchbot IP ranges are allowed. OpenAI's help page names this as the second condition for eligibility, and a 403 response to the bot has the same effect as a robots.txt block.

Wait about 24 hours

OpenAI says it can take about 24 hours from a robots.txt update for its search systems to adjust, as of November 2026. Do not judge the change on the first day. Ask ChatGPT a question your page answers after that window, and check whether your page appears among the cited sources.

Keep the page readable without JavaScript

Put the facts you want cited in the HTML your server sends. Vercel and MERJ found in December 2024 that the major AI crawlers did not run JavaScript, so text that only appears after a script runs may never reach the bot.

A 5-minute check is to view the page source and search for the price.

Making facts visible in raw HTML is core generative engine optimization practice. It removes a reason the bot cannot read your page, even though it cannot force ChatGPT to cite you.

Tools and time needed

You need no paid tool to manage OAI-SearchBot. A text editor for robots.txt, access to your server logs, and your firewall or CDN dashboard cover the whole job. The edit itself takes minutes for one file, and the wait for OpenAI's systems to adjust takes about 24 hours.

Use your server logs to confirm the bot visits. Search the log for OAI-SearchBot, check the IP against the 39 prefixes in OpenAI's searchbot.json file, and note the status code on each hit.

A 200 means the page was served, while a 403 or 429 means a firewall or rate limit stopped the bot.

Use your browser to check what the bot can read. Open view-source on a key page and search for the price, plan name, or fact you want cited.

If it is missing from the source, the bot likely misses it too. Our guide to AI visibility explains how to track whether AI answers mention your brand at all.

OpenAI's help page gives no ranking checklist and no tool for submitting pages. The log check and the source check above are the direct evidence you can gather yourself, and both are free.

How to measure results

Measure OAI-SearchBot in 3 places: your logs, your referrals, and ChatGPT answers. Logs show whether the bot fetches your pages.

Referrals show whether readers arrive. Answers show whether your pages get cited.

Start with logs. Count OAI-SearchBot hits per week, split by status code, and watch for a rise after a robots.txt change.

Steady 200 responses mean the bot can reach your pages. A run of 403 responses means a firewall rule is undoing your allow rule.

Then check referrals. Look in your analytics for traffic from ChatGPT and compare it week by week. Expect small numbers: Cloudflare reported in August 2025 that OpenAI crawled 1,091 pages per referred visit in July 2025.

Then test answers directly. Ask ChatGPT the exact questions your pages answer, once a week, and note whether your page is cited. OpenAI warns that citations can be incomplete, outdated, or incorrect, so open each cited source and check it supports the answer.

Track brand mentions too. A citation is not the only sign that ChatGPT knows your site: answers can name a brand without linking to it. Our guide to brand mentions in AI search explains how to count those mentions.

Index checks sit behind any citation. Our guide to checking whether a backlink is indexed covers the checks that show whether a page is in an index at all.

Common mistakes

  • Blocking GPTBot and OAI-SearchBot together: many sites copy a block list that names both agents. That opts the site out of training and out of ChatGPT search answers at once. Block GPTBot alone if training is the concern.
  • Allowing the bot in robots.txt but blocking its IPs: a firewall or CDN rule that returns 403 to OpenAI's IP ranges has the same effect as a Disallow line. OpenAI's help page names IP access as a second condition for eligibility.
  • Judging the change on day 1: OpenAI says its search systems can take about 24 hours to adjust after a robots.txt update. Check results after that window, not within the first hours.
  • Hiding key facts behind JavaScript: Vercel and MERJ found in December 2024 that the major AI crawlers did not run JavaScript. Prices and facts that load only in the browser may never reach the bot.
  • Expecting a citation as a reward for allowing the bot: OpenAI says placement is not guaranteed and that results use multiple factors. Access makes a page eligible. It does not promise a mention.
  • Trusting the user agent string alone: anyone can copy the OAI-SearchBot string. Check the request IP against the 39 prefixes in OpenAI's searchbot.json file before treating traffic as genuine.

Frequently Asked Questions

No. GPTBot and OAI-SearchBot are separate controls, and OpenAI says each setting is independent of the others.

Blocking GPTBot signals that your content should not be used to train models. ChatGPT search depends on OAI-SearchBot, so your pages stay eligible for search answers while GPTBot is blocked.

OpenAI says it can take about 24 hours from a robots.txt update for its search systems to adjust, as of November 2026. That is the figure OpenAI publishes for search.

If your pages still do not appear after 2 days, recheck both the robots.txt file and your firewall, since a network block can outlast the 24-hour window.

Can OAI-SearchBot see content that loads with JavaScript?

Probably not, based on the best measurement available. Vercel and MERJ reported in December 2024 that none of the major AI crawlers they studied rendered JavaScript, and they named OAI-SearchBot in that finding. OpenAI does not document JavaScript rendering either way, so put key facts in the HTML your server sends.

How do I know a visit is really from OAI-SearchBot?

Check the request IP against OpenAI's published list. The searchbot.json file lists 39 IPv4 prefixes with a creation time of 2 January 2026, and OpenAI links it from its crawler page.

A matching IP plus the OAI-SearchBot user agent is genuine traffic. A matching user agent from another IP is not, since anyone can copy the string.

Does allowing OAI-SearchBot guarantee that ChatGPT will cite my pages?

No. Allowing the bot makes your pages eligible to appear in ChatGPT search, and OpenAI says it ranks results with multiple factors and that placement is not guaranteed.

Relevance, reliability, and the question itself decide which eligible pages get cited. Treat crawler access as the first gate, not as a promise of a citation.

What is the difference between OAI-SearchBot and ChatGPT-User?

OAI-SearchBot crawls automatically on OpenAI's schedule to build the pool for ChatGPT search.

ChatGPT-User visits a single page when a person asks ChatGPT or a Custom GPT to read it. OpenAI says ChatGPT-User does not crawl the web automatically, that robots.txt rules may not apply to it, and that it does not decide whether content may appear in search.

Sources

For what happens after the crawl, see how ChatGPT chooses which sources to cite.

BacklinkPrice Team
Written by
BacklinkPrice Team
Link building and backlink pricing research

The BacklinkPrice Team researches backlink prices, publisher sites and link building tools. We check prices and plans against each provider's own pages and date them, so you know how current the numbers are.

More from BacklinkPrice Team

Related guides