Should You Block GPTBot?

Block GPTBot if you do not want OpenAI to train its models on your content, and keep OAI-SearchBot allowed so your pages can still appear in ChatGPT search answers.
Block GPTBot only if you do not want OpenAI to use your site content to train its models, and keep OAI-SearchBot allowed if you want to appear in ChatGPT search answers. GPTBot is OpenAI's training crawler, and blocking it does not remove your pages from ChatGPT search, because OpenAI uses a separate crawler for search.
Key takeaways
- GPTBot collects content that may train OpenAI models, while OAI-SearchBot decides whether your pages can appear in ChatGPT search answers (OpenAI, 2026).
- OpenAI treats the two settings as independent, so you can disallow GPTBot and allow OAI-SearchBot in the same robots.txt file (OpenAI, 2026).
- GPTBot was the most blocked AI bot in Ahrefs' 2025 count of about 140 million sites, at 5.89%, while only 0.50% of sites named it in an explicit rule.
- Training drove about 80% of AI bot activity in the 12 months to July 2025, while search drove 18%, according to Cloudflare (2025).
- A robots.txt rule is a request, not a lock: Google Search Central says crawlers may ignore it, and a blocked page can still appear in search results (Google, 2025).
What GPTBot is
GPTBot is OpenAI's web crawler for training data. It visits public pages and collects content that may be used to train OpenAI's generative AI foundation models, according to OpenAI's crawler documentation (2026). It is one of several AI crawlers that visit a site, and each one has a separate job.
OpenAI lists 4 user agents in its crawler documentation as of November 2026: GPTBot for training, OAI-SearchBot for ChatGPT search, ChatGPT-User for actions a person starts, and OAI-AdsBot for ad landing pages.
GPTBot identifies itself with a user agent string that ends in GPTBot/1.4 in the example OpenAI publishes, and OpenAI notes the version number may change (OpenAI, 2026).
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
Why blocking GPTBot matters
Blocking GPTBot matters because it is the only robots.txt setting that signals your content should not be used to train OpenAI's models. That choice protects licensing value for publishers and creators, but it gives up any chance that future models learn your brand, products, or views from your own pages.
News publishers block GPTBot far more often than other sites. HasData found in July 2026 that 50.5% of 1,148 news publishers blocked GPTBot, against 7.9% of 9,746 top websites, as summarized by SEO Sandwitch (2026).
The Reuters Institute found in 2024 that 48% of the 15 biggest news sites in each of 10 countries blocked OpenAI's crawlers by the end of 2023, with United States sites at 79% and Mexico and Poland at 20%, as summarized by SEO Sandwitch (2026).
Press Gazette reported in January 2026 that 79% of almost 100 top news sites in the United Kingdom and United States blocked at least one training bot, while 71% also blocked bots used for live search and retrieval.
For most marketing sites, the trade runs the other way. SEO Sandwitch found in November 2026 that only 5% of 37 SEO and marketing sites blocked GPTBot, against 60% of 45 news publishers, because marketing sites want AI visibility and citations in AI answers.
How to block GPTBot in robots.txt
Add a GPTBot group to the robots.txt file at the root of your domain, with User-agent: GPTBot followed by Disallow: /. The edit takes about 5 minutes, and OpenAI says its search systems take about 24 hours to adjust after a robots.txt change (OpenAI, 2026).
- Open your robots.txt file at yourdomain.com/robots.txt and back up the current file before you change it.
- Add a group for GPTBot: User-agent: GPTBot on one line and Disallow: / on the next line.
- Add a second group for OAI-SearchBot with Allow: / if you want your pages to stay eligible for ChatGPT search answers.
- Save the file, then fetch yourdomain.com/robots.txt in a browser and check that both groups appear exactly as written.
- Wait about 24 hours, then check server logs for GPTBot requests and confirm the count falls.
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
You can block only part of a site by replacing Disallow: / with a path such as Disallow: /blog/ or Disallow: /premium/. Use that option when only some sections carry licensed or paywalled content and the rest is public marketing material.
Do not expect a robots.txt rule to stop ChatGPT-User. OpenAI says robots.txt rules may not apply to actions a person starts in ChatGPT, and that agent does not decide how ChatGPT chooses which sources to cite (OpenAI, 2026).
What blocking changes, and what it does not
Blocking GPTBot changes training use only, and it does not change whether your pages can appear in ChatGPT search answers. OpenAI states each setting is independent, and gives the example of allowing OAI-SearchBot while disallowing GPTBot (OpenAI, 2026).
| OpenAI bot | Job | If you block it |
|---|---|---|
| GPTBot | Training data for models | Future models learning from your pages |
| OAI-SearchBot | ChatGPT search answers | Eligibility for ChatGPT search answers |
| ChatGPT-User | Fetches a page a person asks about | Little, since robots.txt may not apply |
| OAI-AdsBot | Checks ad landing pages | Ad checks on pages you submit as ads |
Blocking GPTBot also does not remove content from models already trained, and it does not delete pages from any index. It signals how future training should treat the content you publish from now on.
Cloudflare found in 2025 that GPTBot's share of AI crawling traffic rose from 4.7% in July 2024 to 11.7% in July 2025, and its share of AI-only crawling rose from 11.9% to 28.1% over the same period, in Cloudflare's crawl-to-click analysis.
Cloudflare's 2025 Radar Year in Review found GPTBot produced about 7.5% of verified bot traffic on its network, second only to Googlebot, while AI crawlers as a group produced 20%.
Training drove about 80% of AI bot activity in the 12 months to July 2025, up from 72% a year earlier, while search drove 18% and user actions drove 2%, according to Cloudflare (2025).
A citation in ChatGPT search comes from the search index, not from training data, so allowing OAI-SearchBot is what keeps a page eligible for an AI citation when ChatGPT answers a question.
Tools and time needed
You need access to your robots.txt file, your server logs, and about 30 minutes, and the change itself costs nothing. Most sites edit the file through their host, their content management system, or a developer, and no paid tool is required for the basic block.
- Your robots.txt file, edited at the domain root.
- Server logs or your host log viewer, to confirm GPTBot visits fall.
- Your content delivery network or firewall settings, to check no rule blocks OAI-SearchBot by accident.
- A plain text editor and a backup copy of the old file, so you can undo the change in minutes.
If you want AI search traffic after the change, pair the robots.txt edit with generative engine optimization: clear answers, sourced facts, and pages that load fast for crawlers and readers.
Editing robots.txt costs nothing beyond your own time, as of November 2026, and OpenAI charges no fee for blocking or allowing its crawlers.
How to measure the result
Measure the result in your server logs and in ChatGPT search, not in Google rankings, because GPTBot does not control Google rankings. Check GPTBot request counts by week, check that OAI-SearchBot still fetches pages, and ask ChatGPT a question your page answers to see whether it cites the page.
Cloudflare measured in 2025 that OpenAI crawled about 1,091 pages in July 2025 for each referred visit, with a January to July 2025 average of about 1,438 pages per visit, so expect many crawls and few clicks back, in Cloudflare's crawl-to-click analysis.
Google referrals to news sites fell 9% in March 2025 against January, and 15% in April, in Cloudflare's 2025 data, so track referrals from AI search separately from Google search when you judge the change.
Watch how Google AI Overviews choose sources too, because a GPTBot block does not affect Google's crawlers or its AI Overviews.
- GPTBot requests per week in server logs, before and after the change.
- OAI-SearchBot requests per week, which should stay steady if you allowed it.
- ChatGPT search citations for 5 to 10 of your key pages, checked once a month.
- Referral visits from ChatGPT in analytics, split from Google search traffic.
Common mistakes
- Blocking every OpenAI bot at once: a wildcard block on GPTBot, OAI-SearchBot, and ChatGPT-User removes your pages from ChatGPT search answers as well as training, which most sites do not intend.
- Forgetting the firewall: a robots.txt Allow line does nothing if your host or content delivery network returns a 403 error to OAI-SearchBot, so test the fetch with the real user agent.
- Using a site-wide block for a partial problem: blocking GPTBot on the whole site when only a paid section needs protection gives up training exposure for public pages that could have stayed open.
- Trusting robots.txt alone: HasData found in 2026 that 46.7% of top sites that disallowed GPTBot in robots.txt still served it a page in a live test, as summarized by SEO Sandwitch, so check logs and server rules too.
- Blocking GPTBot and expecting a traffic gain: the block stops training use, and it does not add citations, so pair it with clear, sourced pages if you want brand mentions in AI search and citations.
Frequently Asked Questions
Will blocking GPTBot remove my site from ChatGPT?
No, blocking GPTBot only signals that your content should not be used to train OpenAI models, as of November 2026. ChatGPT search uses OAI-SearchBot, a separate crawler, so keep that bot allowed if you want your pages to appear in ChatGPT search answers. OpenAI states the two settings are independent.
Does blocking GPTBot help my Google rankings?
No, it does not help or hurt Google rankings directly. GPTBot is OpenAI's crawler, not Google's, and Googlebot reads its own robots.txt rules. Keep Googlebot allowed unless you have a separate reason to limit Google crawling, since a GPTBot block changes training use only.
How long does a GPTBot block take to work?
The file edit takes minutes, and OpenAI says its search systems take about 24 hours to adjust after a robots.txt change. Training effects take longer to show, because models train on schedules OpenAI does not publish, so check your logs after a week.
Should a small blog block GPTBot?
Most small blogs should not block it, because their pages gain little protection and lose the chance that future models know their work. Block it only if you sell or license original content, or if a contract limits how others may reuse it.
Can I block GPTBot on some pages only?
Yes. Use a path rule such as Disallow: /members/ under the GPTBot group, and leave the rest of the site open. Check the file after you save it, and confirm in logs that GPTBot stops fetching the closed paths while OAI-SearchBot still fetches the open ones.
Sources
- OpenAI: Overview of OpenAI Crawlers (2026)
- Google Search Central: Robots.txt Introduction and Guide (2025)
- Cloudflare: The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals (2025)
- Cloudflare: The 2025 Cloudflare Radar Year in Review (2025)
- SEO Sandwitch: AI Crawler Blocking Statistics: Who Blocks AI Bots (2026)
- Press Gazette: Eight in ten of world's biggest news websites now block AI training bots (2026)
The BacklinkPrice Team researches backlink prices, publisher sites and link building tools. We check prices and plans against each provider's own pages and date them, so you know how current the numbers are.
More from BacklinkPrice Team




