{"id":725846,"date":"2026-05-06T16:45:16","date_gmt":"2026-05-06T13:45:16","guid":{"rendered":"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/"},"modified":"2026-05-06T16:45:16","modified_gmt":"2026-05-06T13:45:16","slug":"your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/","title":{"rendered":"Your managed WordPress might be blocking AI bots and you can\u2019t see it"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a2400b38026d\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a2400b38026d\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Your_SEO_tools_show_nothing_wrong_But_platform-level_limits_may_be_quietly_blocking_AI_systems_from_accessing_and_citing_your_content\" >Your SEO tools show nothing wrong. But platform-level limits may be quietly blocking AI systems from accessing and citing your content.<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#What_7_days_of_Cloudflare_logs_showed\" >What 7 days of Cloudflare logs showed<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Where_we_looked_first_and_why_we_were_wrong\" >Where we looked first, and why we were wrong<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Suspect_1_Solid_Securitys_HackRepair_default_ban_list\" >Suspect 1: Solid Security\u2019s HackRepair default ban list<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Suspect_2_Solid_Securitys_other_firewall_subsystems\" >Suspect 2: Solid Security\u2019s other firewall subsystems<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Suspect_3_Sucuri_Cloud_WAF\" >Suspect 3: Sucuri Cloud WAF<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Suspect_4_Cloudflare_itself\" >Suspect 4: Cloudflare itself<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#The_reproduction_test_that_changed_everything\" >The reproduction test that changed everything<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#The_bot-by-bot_fingerprint\" >The bot-by-bot fingerprint<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Why_this_is_hard_to_find\" >Why this is hard to find<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#What_WP_Engine_confirmed_when_I_asked\" >What WP Engine confirmed when I asked<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#WP_Engine_appears_to_be_the_outlier_here\" >WP Engine appears to be the outlier here<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#How_to_check_whether_its_happening_to_you\" >How to check whether it\u2019s happening to you<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Step_1_Reproduce_with_curl_a_command-line_tool_that_fetches_URLs\" >Step 1: Reproduce with curl (a command-line tool that fetches URLs)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Step_2_Identify_your_host\" >Step 2: Identify your host<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Step_3_Check_what_the_host_actually_controls\" >Step 3: Check what the host actually controls\u00a0<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#What_to_do_once_you_know\" >What to do once you know<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Escalate_to_your_hosts_product_engineering\" >Escalate to your host\u2019s product engineering<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-19\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Allowlist_via_the_customer-controllable_Web_Rules_Engine\" >Allowlist via the customer-controllable Web Rules Engine<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-20\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Move_to_a_host_that_doesnt_impose_this\" >Move to a host that doesn\u2019t impose this<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-21\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Accept_the_block_as_a_deliberate_policy\" >Accept the block as a deliberate policy<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-22\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#The_citation_correlation\" >The citation correlation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-23\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Caveats\" >Caveats<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-24\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#What_you_should_do_next\" >What you should do next<\/a><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><ul class='ez-toc-list-level-5' ><li class='ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-25\" href=\"https:\/\/buradabiliyorum.com\/en\/your-managed-wordpress-might-be-blocking-ai-bots-and-you-cant-see-it\/#Topics_on_this_page\" >Topics on this page<\/a><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h2 class=\"subhead\" itemprop=\"alternativeHeadline\"><span class=\"ez-toc-section\" id=\"Your_SEO_tools_show_nothing_wrong_But_platform-level_limits_may_be_quietly_blocking_AI_systems_from_accessing_and_citing_your_content\"><\/span>Your SEO tools show nothing wrong. But platform-level limits may be quietly blocking AI systems from accessing and citing your content.<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><\/p>\n<div class=\"bialty-container\">\n<p>Everything looked normal in the SEO data. Google Search Console, traffic, and indexing \u2014 no red flags. Then I opened Scrunch, our AI citation monitoring tool, and looked at platform-by-platform presence for searchinfluence.com over the prior 30 days:<\/p>\n<ul class=\"wp-block-list\">\n<li>Google AI Mode: 37.8%<\/li>\n<li>Copilot: 22.2%<\/li>\n<li>Google Gemini: 16.3%<\/li>\n<li>ChatGPT: 9.6%<\/li>\n<li>Perplexity: 7.8%<\/li>\n<li><strong>Claude: 0.0%<\/strong><\/li>\n<li><strong>Meta AI: 0.0%<\/strong><\/li>\n<\/ul>\n<p>Two platforms at zero. Every crawler reads the same site, so content quality and topical authority can\u2019t account for that gap. They\u2019re identical for every platform on the list. <\/p>\n<p>What varies is access \u2014 whether each platform\u2019s crawler is allowed in. Nothing else explains how Google AI Mode hits 37.8% while Claude lands at 0%. So I opened the logs.<\/p>\n<h2 id=\"what-7-days-of-cloudflare-logs-showed\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_7_days_of_Cloudflare_logs_showed\"><\/span>What 7 days of Cloudflare logs showed<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Seven days of Cloudflare data (April 4-10) for searchinfluence.com revealed 29,099 bot requests, 65.8% of them AI bots. Here\u2019s a per-bot share of those requests rate-limited (HTTP 429, \u201ctoo many requests\u201d), broken out by bot user-agent (UA, the identifier each request sends):<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img fetchpriority=\"high\" decoding=\"async\" width=\"2048\" height=\"1175\" http: alt=\"chart-waf-rate-limit-by-bot\" class=\"wp-image-476515\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-waf-rate-limit-by-bot-scaled.png.webp 2048w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-waf-rate-limit-by-bot-768x441.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-waf-rate-limit-by-bot-1536x881.png 1536w\" data-lazy-sizes=\"(max-width: 2048px) 100vw, 2048px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-waf-rate-limit-by-bot-scaled.png.webp\"><img fetchpriority=\"high\" decoding=\"async\" width=\"2048\" height=\"1175\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-waf-rate-limit-by-bot-scaled.png.webp\" alt=\"chart-waf-rate-limit-by-bot\" class=\"wp-image-476515\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-waf-rate-limit-by-bot-scaled.png.webp 2048w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-waf-rate-limit-by-bot-768x441.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-waf-rate-limit-by-bot-1536x881.png 1536w\" sizes=\"(max-width: 2048px) 100vw, 2048px\"><\/figure>\n<\/div>\n<ul class=\"wp-block-list\">\n<li>Amazonbot: 51% rate-limited<\/li>\n<li>ClaudeBot: 29%<\/li>\n<li>GPTBot: 29%<\/li>\n<li>Bytespider: 61% blocked (different mechanism: 403\/5xx, not 429)<\/li>\n<li>ChatGPT-User: 0%<\/li>\n<li>PerplexityBot: 0%<\/li>\n<\/ul>\n<p>The split isn\u2019t random. Training crawlers, the ones that pull whole sites in big bursts, get throttled. User-facing crawlers, the ones that fire human-paced requests during a live user query, don\u2019t.<\/p>\n<p>For context: <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/blog.cloudflare.com\/crawlers-click-ai-bots-training\/\" target=\"_blank\" rel=\"noopener\">Cloudflare\u2019s Q1 2026 crawl-to-referral analysis<\/a> shows ClaudeBot makes 20,583 crawl requests for every referral it sends back.\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>GPTBot: 1,255 to 1.\u00a0<\/li>\n<li>Perplexity: 111 to 1.\u00a0<\/li>\n<li>Google: 5 to 1.\u00a0<\/li>\n<\/ul>\n<p>AI training crawlers take far more than they give back, so it makes sense that hosting infrastructure has started fighting back. Whether that\u2019s the right fight for your site is a separate question.<\/p>\n<p id=\"h-\">The 429s in our logs were being passed through Cloudflare with a cache status of dynamic or bypass. So I wrote them off as downstream of Cloudflare, must be a web <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lication firewall (WAF) or security plugin. That assumption sent me down a multi-hour rabbit hole through the wrong layers.<\/p>\n<div style=\"background: radial-gradient(circle at 30% 40%, rgba(184, 111, 255, 0.15), rgba(0, 169, 255, 0.15) 40%, #CDE8FD 70%); padding: 30px; width: 100%; max-width: 802px; color: #000000 !important; font-family: Arial, sans-serif; margin: 25px 0 30px 0; border-radius: 8px; box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1); position: relative; box-sizing: border-box;\">\n<div style=\"width: 100%; max-width: 100%; margin-bottom: 20px; text-align: left; padding-right: 20px; box-sizing: border-box;\">\n<div id=\"semrush-one-headline\" class=\"headline-responsive\" style=\"font-family: Oswald, sans-serif; font-size: 30px; font-weight: normal; margin: 0; color: #000000 !important; line-height: 1.2;\">\n        Your customers search everywhere. Make sure your brand <span style=\"background: linear-gradient(90deg, #D56EFE 0%, #068EF8 51%); -webkit-background-clip: text; -webkit-text-fill-color: transparent; background-clip: text;\">shows up<\/span>.\n      <\/div>\n<p id=\"semrush-one-subhead\" style=\"font-family: Roboto, sans-serif; font-size: 18px; font-weight: 300; line-height: 25px; margin: 12px 0 0 0; color: #000000 !important;\">\n        The SEO toolkit you know, plus the AI visibility data you need.\n      <\/p>\n<\/p><\/div>\n<div style=\"margin-bottom: 15px;\">\n      <span id=\"semrush-one-cta\" style=\"display: inline-block; background-color: #FF642D; color: white; height: 44px; border: none; border-radius: 5px; cursor: pointer; font-size: 16px; padding: 0 24px; font-weight: bold; white-space: nowrap; box-sizing: border-box; text-decoration: none; line-height: 44px;\">Start Free Trial<\/span>\n    <\/div>\n<div style=\"font-size: 12px;\">\n<div style=\"font-family: Roboto, sans-serif; font-weight: 300; color: #000000; margin-bottom: 4px;\">Get started with<\/div>\n<p>      <img loading=\"lazy\" width=\"400\" height=\"52\" decoding=\"async\" http: alt=\"Semrush One Logo\" style=\"height: 16px; width: auto; display: block;\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/semrush-one.webp\"><img loading=\"lazy\" width=\"400\" height=\"52\" decoding=\"async\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/semrush-one.webp\" alt=\"Semrush One Logo\" style=\"height: 16px; width: auto; display: block;\">\n    <\/div>\n<\/p><\/div>\n<style>\n  @media (max-width: 768px) {\n    .headline-responsive {\n      font-size: 30px !important;\n      line-height: 1.3 !important;\n    }\n  }\n<\/style>\n<\/p>\n<h2 id=\"where-we-looked-first-and-why-we-were-wrong\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Where_we_looked_first_and_why_we_were_wrong\"><\/span>Where we looked first, and why we were wrong<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1151\" http: alt=\"chart-request-path-architecture\" class=\"wp-image-476518\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-scaled.png.webp 2048w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-768x431.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-1536x863.png 1536w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-1200x675.png.webp 1200w\" data-lazy-sizes=\"(max-width: 2048px) 100vw, 2048px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-scaled.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1151\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-scaled.png.webp\" alt=\"chart-request-path-architecture\" class=\"wp-image-476518\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-scaled.png.webp 2048w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-768x431.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-1536x863.png 1536w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-request-path-architecture-1200x675.png.webp 1200w\" sizes=\"auto, (max-width: 2048px) 100vw, 2048px\"><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Suspect_1_Solid_Securitys_HackRepair_default_ban_list\"><\/span>Suspect 1: Solid Security\u2019s HackRepair default ban list<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>A WordPress security plugin we use for hardening, with a built-in bot UA blocklist. Toggled it off, ran a 24-hour before\/after on per-bot 429 counts. No change.\u00a0<\/p>\n<p>Two bots even spiked higher in the post-toggle window. Coincidental crawl bursts, not a regression. Ruled out.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Suspect_2_Solid_Securitys_other_firewall_subsystems\"><\/span>Suspect 2: Solid Security\u2019s other firewall subsystems<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>24,538 firewall log entries over 30 days. Every single one was a <code>\/wp-login.php<\/code> brute-force lockout. Zero entries for ClaudeBot, GPTBot, or Amazonbot. Rules empty. IP Management clean. Ruled out.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Suspect_3_Sucuri_Cloud_WAF\"><\/span>Suspect 3: Sucuri Cloud WAF<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>SI has a Sucuri subscription. Logged into the portal and saw warnings across every service column (Monitoring, Firewall\/CDN, SSL, Backups). A dig and curl confirmed why: DNS resolved to Cloudflare ranges, and response headers showed no x-sucuri-id. Sucuri was never in the request path. The subscription existed; the activation never happened. Ruled out.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Suspect_4_Cloudflare_itself\"><\/span>Suspect 4: Cloudflare itself<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Originally written off because cache-status was dynamic\/bypass. That inference was sloppy: Cloudflare can return 429 from rate-limit rules with the same cache-status. Going back to the right view (<em>Security \u2192 Analytics \u2192 Events<\/em> tab, filtered by ClaudeBot UA, last 24 hours): zero events. Cloudflare took no security action on ClaudeBot in 24 hours while passing through 608 ClaudeBot 429s. Ruled out.<\/p>\n<p>At that point, we were out of suspects on layers we could see.<\/p>\n<h2 id=\"the-reproduction-test-that-changed-everything\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_reproduction_test_that_changed_everything\"><\/span>The reproduction test that changed everything<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>We ran 60 fast curl requests with a ClaudeBot UA against three different paths. 60 x 429 every time. Control runs: same paths, browser UA \u2192 60 x 200 (HTTP \u201cOK\u201d). Same paths, Googlebot UA \u2192 60 x 200. The block was unambiguously UA-based \u2014 not path-based, not rate-based.<\/p>\n<p>The headers gave it away. A single curl -I showed x-powered-by: WP Engine. We were on a managed host, and the block was firing from a layer that hadn\u2019t been on the suspect list: the host\u2019s own platform infrastructure, sitting between Cloudflare and WordPress. The hosting platform itself.<\/p>\n<h2 id=\"the-botbybot-fingerprint\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_bot-by-bot_fingerprint\"><\/span>The bot-by-bot fingerprint<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Once we knew which question to ask, we ran the rest of the AI bot UA list through the same curl harness.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1017\" http: alt=\"chart-bot-fingerprint\" class=\"wp-image-476526\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-bot-fingerprint-scaled.png.webp 2048w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-bot-fingerprint-768x381.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-bot-fingerprint-1536x763.png 1536w\" data-lazy-sizes=\"(max-width: 2048px) 100vw, 2048px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-bot-fingerprint-scaled.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1017\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-bot-fingerprint-scaled.png.webp\" alt=\"chart-bot-fingerprint\" class=\"wp-image-476526\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-bot-fingerprint-scaled.png.webp 2048w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-bot-fingerprint-768x381.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-bot-fingerprint-1536x763.png 1536w\" sizes=\"auto, (max-width: 2048px) 100vw, 2048px\"><\/figure>\n<\/div>\n<figure class=\"wp-block-table\">\n<table>\n<thead>\n<tr>\n<th><strong>Bot UA<\/strong><\/th>\n<th><strong>Result<\/strong><\/th>\n<th><strong>Status<\/strong><\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>ClaudeBot<\/td>\n<td>60\/60 x 429<\/td>\n<td>Blocked<\/td>\n<\/tr>\n<tr>\n<td>GPTBot<\/td>\n<td>8\/10 x 429, 2\/10 cached (200)<\/td>\n<td>Blocked<\/td>\n<\/tr>\n<tr>\n<td>Amazonbot<\/td>\n<td>10\/10 x 429<\/td>\n<td>Blocked<\/td>\n<\/tr>\n<tr>\n<td>Bytespider<\/td>\n<td>10\/10 x 520<\/td>\n<td>Blocked (520 is a Cloudflare-specific error: origin returned an invalid response, possibly IP-blackholed)<\/td>\n<\/tr>\n<tr>\n<td>anthropic-ai (older Anthropic UA)<\/td>\n<td>10\/10 x 200<\/td>\n<td>Not blocked<\/td>\n<\/tr>\n<tr>\n<td>CCBot (Common Crawl)<\/td>\n<td>10\/10 x 200<\/td>\n<td>Not blocked<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n<p>Two findings:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>The blocklist is dated:<\/strong> It targets the AI training crawler set as of mid-2024. The older anthropic-ai UA is allowed. CCBot, the Common Crawl bot that feeds many LLM training pipelines, is allowed. If the intent is \u201cno LLM training data,\u201d this gap defeats it. Scrapers can use CCBot\u2019s UA, or Common Crawl can pull the site directly, and the data ends up in training sets anyway. The named-bot list is a fence with a gate left open.<\/li>\n<li><strong>Cached responses serve through the block:<\/strong> WP Engine\u2019s edge cache returns cached pages to ClaudeBot just fine (x-cache: HIT in the headers). Cache-miss requests hit the origin handler and get 429. This explains the Cloudflare data exactly: in 24 hours, 1,054 ClaudeBot requests returned 200 (cache hits) and 608 returned 429 (cache misses). Same UA, same site, two outcomes.<\/li>\n<\/ul>\n<p>It\u2019s worth flagging that ~100% of our 24-hour \u201cClaudeBot\u201d traffic came from a single Microsoft\/Azure IP (AS8075, Microsoft\u2019s network), not Anthropic\u2019s published AWS ranges. Almost certainly, it\u2019s a spoofed UA: a scraper on Azure pretending to be ClaudeBot. A meaningful slice of \u201cAI crawler 429s\u201d in WAF reports may be appropriate for blocking imposter traffic, not legitimate Anthropic crawl.<\/p>\n<h2 id=\"why-this-is-hard-to-find\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Why_this_is_hard_to_find\"><\/span>Why this is hard to find<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Start with what WP Engine itself says about its firewall. From their <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/wpengine.com\/support\/wp-engines-security-environment\/\" target=\"_blank\" rel=\"noopener\">support page on the security environment<\/a>:\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>\u201cFurther information cannot be provided around our firewall, as this can compromise its secure integrity.\u201d\u00a0<\/li>\n<\/ul>\n<p>That\u2019s the company\u2019s own statement, verbatim. Whatever the rules are, customers don\u2019t get to see them.<\/p>\n<p>Their <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/wpengine.com\/blog\/wp-engine-2025-in-review\/\">2025 Year in Review<\/a> reports 75 billion bot requests mitigated via Cloudflare-powered bot management. No documented user portal control opts you out per-site or per-bot. I checked every customer-facing setting that could plausibly fire AI bot 429s:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Utilities \u2192 Redirect bots:<\/strong> Off (default).<\/li>\n<li><strong>Web rules:<\/strong> Empty.<\/li>\n<li><strong>Robots.txt setting:<\/strong> Not customized. Live \/robots.txt only disallows a few specific PDFs.<\/li>\n<\/ul>\n<p>All clean. The block is somewhere customers can\u2019t reach.<\/p>\n<p>A few more reasons it\u2019s invisible:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>It returns 429, not 403:<\/strong> Returning \u201cforbidden\u201d can get a site flagged by search engines as a site-wide failure, so 429 is the safer choice. But 429 reads as \u201crate limit\u201d in every WAF analytics tool, which sends investigators chasing rate-limit configurations at the wrong layer.<\/li>\n<li><strong>It fires below the WAF plugins:<\/strong> Wordfence, Sucuri, and Solid Security all log at the WordPress application layer. WP Engine\u2019s block fires at the platform edge, before the request reaches WordPress. Plugin logs show nothing.<\/li>\n<li><strong>It fires below customer Cloudflare, too:<\/strong> WP Engine <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.itbrew.com\/stories\/2025\/05\/20\/inside-wp-engine-s-playbook-for-warding-off-ai-bots\" target=\"_blank\" rel=\"noopener\">runs its own Cloudflare-backed bot<\/a> management at the hosting edge. That\u2019s a separate Cloudflare layer behind your own Cloudflare zone. Events fired there don\u2019t appear in your Cloudflare dashboard.<\/li>\n<li><strong>WP Engine\u2019s billing already accounts for the block:<\/strong> They <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/wpengine.com\/blog\/website-traffic-trends-findings\/\">exclude \u201csuspected bots\u201d from billable metrics<\/a>. From a hosting-cost perspective, the customer benefits. From a GEO\/AEO perspective, the customer pays in citation absence, without ever knowing they signed up.<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_WP_Engine_confirmed_when_I_asked\"><\/span>What WP Engine confirmed when I asked<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>After several rounds of canned auto-replies, I reached a live agent. The relevant exchanges:<\/p>\n<p>On the policy itself:\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>\u201cWP Engine does enforce platform\u2011wide rate limiting on certain high\u2011impact bots to protect overall server performance, and that part can\u2019t be selectively disabled per bot.\u201d<\/li>\n<\/ul>\n<p>On whether the customer-facing Web Rules Engine could route around it:\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>\u201cAllowing AI bot IPs via Web Rules Engine does not override WP Engine\u2019s platform-wide rate limiting rules, which operate at the infrastructure level.\u201d<\/li>\n<\/ul>\n<p>On whether the SEO downside was acknowledged anywhere internally:\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>\u201cThe documentation acknowledges that blocking or rate limiting bots like Amazonbot and similar user agents can impact their crawling and indexing\u2026 It emphasizes balancing bot management with SEO considerations and suggests customers be empathetic as many did not configure these bots themselves.\u201d<\/li>\n<\/ul>\n<p>Read that last bit twice. The internal framing assumes the customer is being protected from bots they didn\u2019t ask for. For agencies, content sites, B2B SaaS, and anyone whose growth depends on AI search citations, the assumption inverts. Those bots are the audience the customer is trying to reach.<\/p>\n<p>There\u2019s an escalation path:\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>\u201cIf you have an exceptional use case or need a bot to behave differently than the platform defaults allow, we can escalate it to ProdEng (product engineering) for review.\u201d\u00a0<\/li>\n<\/ul>\n<p>So the policy isn\u2019t immutable. It\u2019s just not a self-service setting.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"WP_Engine_appears_to_be_the_outlier_here\"><\/span>WP Engine appears to be the outlier here<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>We assumed every managed host did this. Public record on the other three top-managed WordPress hosts contradicts that:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Kinsta\u2019s CTO said in March 2026<\/strong> that they <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/webhosting.today\/2026\/03\/20\/ai-crawlers-are-eating-your-bandwidth-how-hosting-companies-are-fighting-back\/\" target=\"_blank\" rel=\"noopener\">will not block at the platform level<\/a> and will not bill for bot bandwidth. Their <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/kinsta.com\/docs\/wordpress-hosting\/mykinsta-tools\/wordpress-tools-bot-protection\/\" target=\"_blank\" rel=\"noopener\">Bot Protection feature<\/a> is opt-in, with four customer-controllable levels.<\/li>\n<li><strong>Pressable<\/strong> explicitly states in <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/pressable.com\/knowledgebase\/disallowing-gptbots-from-accessing-your-site-content\/\" target=\"_blank\" rel=\"noopener\">its knowledge base<\/a>: \u201cPressable does not currently disallow these bots by default.\u201d Customer manages it via robots.txt.<\/li>\n<li><strong>Pantheon<\/strong> explicitly <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/docs.pantheon.io\/bots-and-indexing\" target=\"_blank\" rel=\"noopener\">states<\/a>: \u201cWe do not block identified bot traffic from entering the platform.\u201d They detect and exclude bots from billing only.<\/li>\n<\/ul>\n<p>Outside managed WP, the closest analog is SiteGround, which <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.siteground.com\/kb\/ai-crawlers-siteground-servers\/\" target=\"_blank\" rel=\"noopener\">blocks training crawlers<\/a> by default but is more transparent about the policy and distinguishes training bots from user-action bots.<\/p>\n<p>One wrinkle: Flywheel, a managed WP host owned by WP Engine since 2019, has <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/getflywheel.com\/wordpress-support\/all-about-robots-txt-on-flywheel\/\" target=\"_blank\" rel=\"noopener\">no documented AI bot block<\/a>. Same parent company, two products, two different stated policies. Not a corporate-wide stance. A product-level decision specific to WP Engine.<\/p>\n<p>Caveat on the comparison: we confirmed WP Engine\u2019s block empirically with curl. We didn\u2019t run the same diagnostic against Kinsta, Pressable, or Pantheon. What we have for them is their public documentation, which is reliable but not the same as a live test.\u00a0<\/p>\n<p>The precise claim: based on what each host publicly discloses, WP Engine appears to be the only top-tier managed WP host with a default-on, non-disableable platform-level AI bot block.<\/p>\n<p>The question shifts. It\u2019s not \u201cAre other hosts doing this?\u201d It\u2019s \u201cWhy is WP Engine, and apparently only WP Engine, doing it this way?\u201d<\/p>\n<h2 id=\"how-to-check-whether-its-happening-to-you\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_check_whether_its_happening_to_you\"><\/span>How to check whether it\u2019s happening to you<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The standard audit advice for your WAF logs doesn\u2019t catch this. Below are three steps that don\u2019t require root access.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-step-1-reproduce-with-curl-a-command-line-tool-that-fetches-urls\"><span class=\"ez-toc-section\" id=\"Step_1_Reproduce_with_curl_a_command-line_tool_that_fetches_URLs\"><\/span>Step 1: Reproduce with curl (a command-line tool that fetches URLs)<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<pre class=\"wp-block-code\"><code>for i in $(seq 1 30); do\n  curl -sI -A \"ClaudeBot\/1.0 (+https:\/\/www.anthropic.com\/claudebot)\" \\\n    \"https:\/\/yourdomain.com\/\" \\\n    -o \/dev\/null -w \"%{http_code}\\n\"\n  sleep 0.05\ndone | sort | uniq -c<\/code><\/pre>\n<p>Then run the same loop with a Mozilla\/5.0 \u2026 browser UA. If the browser run returns 200s and the ClaudeBot run returns 429s, the block is UA-based and someone in your stack is doing it. If both return the same code, you don\u2019t have this problem.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_2_Identify_your_host\"><\/span>Step 2: Identify your host<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Run curl -I https:\/\/yourdomain.com\/ and look at the response headers for x-powered-by or server. They often name the host (WP Engine, Pressable, Kinsta, etc.). If your host is unmanaged or self-hosted, this article likely doesn\u2019t apply. Investigate your WAF instead.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_3_Check_what_the_host_actually_controls\"><\/span>Step 3: Check what the host actually controls\u00a0<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>For WP Engine specifically, confirm <em>Utilities &gt; Redirect Bots<\/em> is off and that Web Rules has no AI UA blocks, then open a support ticket. Here\u2019s recommended wording:\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>\u201cWe\u2019ve reproduced via curl that requests with ClaudeBot\/GPTBot\/Amazonbot user-agent strings receive HTTP 429 responses for cache-miss requests on our environment. Cloudflare and our security plugins are not the source. Is this WP Engine\u2019s platform-level AI crawler mitigation? Can it be disabled or scoped per-bot for our environment?\u201d<\/li>\n<\/ul>\n<p>For other hosts, the equivalent path is their portal\u2019s security section first, then a support ticket with the same evidence.<\/p>\n<h2 id=\"what-to-do-once-you-know\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_to_do_once_you_know\"><\/span>What to do once you know<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Four real options, in order of effort.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Escalate_to_your_hosts_product_engineering\"><\/span>Escalate to your host\u2019s product engineering<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>WP Engine\u2019s support agent named an \u201cexceptional use case\u201d escalation path. The policy isn\u2019t immutable; it\u2019s just not a self-service toggle. SEO and AI search visibility is exactly the kind of case that the escalation path is built for.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Allowlist_via_the_customer-controllable_Web_Rules_Engine\"><\/span>Allowlist via the customer-controllable Web Rules Engine<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>WREn lets you allowlist UAs at the site level, but the support agent confirmed it doesn\u2019t override the platform rules. It\u2019s useful for the bots <strong>not<\/strong> on the platform list (CCBot, anthropic-ai), but not a fix for the ones that are.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Move_to_a_host_that_doesnt_impose_this\"><\/span>Move to a host that doesn\u2019t impose this<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>A nuclear option, but worth costing out if AI search visibility is a strategic priority and ProdEng escalation goes nowhere. Kinsta\u2019s and Pressable\u2019s documented stances both leave AI crawler access to the customer.<\/p>\n<p>And to be clear: AI search visibility absolutely should be a strategic priority right now. ChatGPT alone handles billions of queries a week, and the answers cite a small set of sources. If your category is being decided in those answers and your site can\u2019t be crawled, you don\u2019t get cited. <\/p>\n<p>There is no \u201cI\u2019ll just rank later\u201d backup plan, because the citation set hardens fast. Treating AI access as optional in 2026 is the same call as treating organic search as optional in 2008. It worked for a while. Then it didn\u2019t.<\/p>\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Accept_the_block_as_a_deliberate_policy\"><\/span>Accept the block as a deliberate policy<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Some companies will conclude that staying out of AI training data is the right call. The honest version: tell the team that\u2019s what\u2019s happening, factor it into AI-search expectations, stop running GEO\/AEO audits that score you on missing citations you weren\u2019t going to get anyway.<\/p>\n<p>The wrong move is to keep running the WAF audit playbook and concluding that nothing\u2019s wrong. The block fires invisibly, and the citation\u2019s absence shows up months later in dashboards that no one connects back to it.<\/p>\n<h2 id=\"the-citation-correlation\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"The_citation_correlation\"><\/span>The citation correlation<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1193\" http: alt=\"chart-access-vs-citation\" class=\"wp-image-476535\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-access-vs-citation-scaled.png.webp 2048w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-access-vs-citation-768x447.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-access-vs-citation-1536x895.png 1536w\" data-lazy-sizes=\"(max-width: 2048px) 100vw, 2048px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-access-vs-citation-scaled.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1193\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-access-vs-citation-scaled.png.webp\" alt=\"chart-access-vs-citation\" class=\"wp-image-476535\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-access-vs-citation-scaled.png.webp 2048w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-access-vs-citation-768x447.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/chart-access-vs-citation-1536x895.png 1536w\" sizes=\"auto, (max-width: 2048px) 100vw, 2048px\"><\/figure>\n<\/div>\n<ul class=\"wp-block-list\">\n<li>Googlebot ~100% access \u2192 Google AI Mode 37.8% citation presence<\/li>\n<li>GPTBot 54% access \u2192 ChatGPT 9.6%<\/li>\n<li>PerplexityBot 100% access \u2192 Perplexity 7.8%<\/li>\n<li>ClaudeBot 57% access \u2192 Claude 0.0%<\/li>\n<\/ul>\n<p>The platform-by-platform split in citations matches the platform-by-platform split in crawl access. Where the bot can read the site, the AI cites it at meaningful rates. Where the bot is blocked, citation presence collapses.<\/p>\n<p>Suggestive, not proof: 7-day correlation on a single site, no controlled before\/after. Part 2 publishes the post-fix numbers if we get the block lifted (or move hosts). The intuition: crawl access is the floor; content quality, topical authority, and freshness are the ceiling. If the bot can\u2019t read you, the ceiling doesn\u2019t matter.<\/p>\n<p>Perplexity is the wrinkle: 100% access, 7.8% citation. Full access alone doesn\u2019t guarantee citation. But the <em>absence<\/em> of access (Claude at 0%) is decisive.<\/p>\n<h2 id=\"caveats\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Caveats\"><\/span>Caveats<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<ul class=\"wp-block-list\">\n<li><strong>Single-site case study:<\/strong> The diagnostic <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a>izes; the specific numbers don\u2019t.<\/li>\n<li><strong>AI citation is multi-factor:<\/strong> Content quality, topical authority, entity coverage, freshness, schema, brand recognition: all of those matter. Crawl access is the floor, not the whole <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/game\/\" data-internallinksmanager029f6b8e52c=\"7\" title=\"Game\" target=\"_blank\" rel=\"noopener\">game<\/a>.<\/li>\n<li><strong>Bot UAs can be spoofed:<\/strong> Roughly 100% of our \u201cClaudeBot\u201d traffic was from a non-Anthropic IP. The host-level block is doing the right thing for those impostors.<\/li>\n<li><strong>AI bots don\u2019t fully respect crawl-delay:<\/strong> <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.inmotionhosting.com\/blog\/rate-limiting-ai-crawler-bots-modsecurity\/\" target=\"_blank\" rel=\"noopener\">InMotion\u2019s coverage<\/a> is a good reference: GPTBot and ClaudeBot only partially honor crawl-delay in robots.txt, so the 429 is one of the few signals they actually act on. That\u2019s a feature, until they improve crawl-delay compliance.<\/li>\n<li><strong>WP Engine\u2019s defaults aren\u2019t malicious:<\/strong> They\u2019re protecting customers who didn\u2019t ask for AI bot traffic. The opacity is the issue, not the intent. Customers who do want the traffic should have a way to say so without escalating to product engineering.<\/li>\n<\/ul>\n<div style=\"background: radial-gradient(circle at 30% 40%, rgba(184, 111, 255, 0.15), rgba(0, 169, 255, 0.15) 40%, #CDE8FD 70%); padding: 30px; width: 100%; max-width: 802px; color: #000000 !important; font-family: Arial, sans-serif; margin: 25px 0 30px 0; border-radius: 8px; box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1); position: relative; box-sizing: border-box;\">\n<div style=\"width: 100%; max-width: 100%; margin-bottom: 20px; text-align: left; padding-right: 20px; box-sizing: border-box;\">\n<div id=\"semrush-one-headline-bottom\" class=\"headline-responsive\" style=\"font-family: Oswald, sans-serif; font-size: 30px; font-weight: normal; margin: 0; color: #000000 !important; line-height: 1.2;\">\n        See the <span style=\"background: linear-gradient(90deg, #D56EFE 0%, #068EF8 51%); -webkit-background-clip: text; -webkit-text-fill-color: transparent; background-clip: text;\">complete picture<\/span> of your search visibility.\n      <\/div>\n<p id=\"semrush-one-subhead-bottom\" style=\"font-family: Roboto, sans-serif; font-size: 18px; font-weight: 300; line-height: 25px; margin: 12px 0 0 0; color: #000000 !important;\">\n        Track, optimize, and win in Google and AI search from one platform.\n      <\/p>\n<\/p><\/div>\n<div style=\"margin-bottom: 15px;\">\n      <span id=\"semrush-one-cta-bottom\" style=\"display: inline-block; background-color: #FF642D; color: white; height: 44px; border: none; border-radius: 5px; cursor: pointer; font-size: 16px; padding: 0 24px; font-weight: bold; white-space: nowrap; box-sizing: border-box; text-decoration: none; line-height: 44px;\">Start Free Trial<\/span>\n    <\/div>\n<div style=\"font-size: 12px;\">\n<div style=\"font-family: Roboto, sans-serif; font-weight: 300; color: #000000; margin-bottom: 4px;\">Get started with<\/div>\n<p>      <img loading=\"lazy\" width=\"400\" height=\"52\" decoding=\"async\" http: alt=\"Semrush One Logo\" style=\"height: 16px; width: auto; display: block;\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/semrush-one.webp\"><img loading=\"lazy\" width=\"400\" height=\"52\" decoding=\"async\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/semrush-one.webp\" alt=\"Semrush One Logo\" style=\"height: 16px; width: auto; display: block;\">\n    <\/div>\n<\/p><\/div>\n<style>\n  @media (max-width: 768px) {\n    .headline-responsive {\n      font-size: 30px !important;\n      line-height: 1.3 !important;\n    }\n  }\n<\/style>\n<\/p>\n<h2 id=\"what-you-should-do-next\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_you_should_do_next\"><\/span>What you should do next<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>If you\u2019re on WP Engine, run the diagnostic above. If the curl reproduction shows the same pattern, you\u2019ve got the same issue. Open a ticket and see where that goes, or switch providers.<\/p>\n<p>If you\u2019re on a different managed host, run it anyway. The diagnostic takes three minutes.<\/p>\n<p>If you\u2019re spending months on content updates, schema markup, and llms.txt files while a default-on platform setting is silently blocking the crawlers you\u2019re trying to reach, you\u2019re optimizing the ceiling of a building with no floor.<\/p>\n<p><em>Full disclosure on method: An AI assistant (Claude) ran the curl tests, parsed headers, and walked the architecture with me. Where this piece says \u201cwe\u201d tested or reproduced something, that\u2019s me plus the AI. Where it says \u201cI,\u201d it was me directly: portal logins, the WP Engine support chat.<\/em><\/p>\n<div class=\"ttd-topics-display\">\n<div class=\"ttd-topics-content\">\n<h5><span class=\"ez-toc-section\" id=\"Topics_on_this_page\"><\/span>Topics on this page<span class=\"ez-toc-section-end\"><\/span><\/h5>\n<div class=\"ttd-topics-links\">WP EngineWordPressCloudflareWeb application firewallArtificial intelligencePerplexity AIGoogle Search ConsoleGoogleMicrosoft AzureChatGPTHTTPGeminiSearch engine optimizationCommon Crawl FoundationAnthropicAmerican Eagle OutfittersTransport Layer SecurityCCBotLarge language modelMicrosoftGooglebotSoftware as a serviceMozillaURLGPTBotWeb crawlerWeb hosting service<\/div>\n<\/div>\n<div class=\"ttd-topics-show-extra-button\">+22 more<\/div>\n<\/div>\n<\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" target=\"_blank\" >Technology<\/a><\/span> category.<\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/searchengineland.com\/managed-wordpress-blocking-ai-bots-476510\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Your SEO tools show nothing wrong. But platform-level limits may be quietly blocking AI systems from accessing and citing your content. Everything looked normal in the SEO data. Google Search Console, traffic, and indexing \u2014 no red flags. Then I opened Scrunch, our AI citation monitoring tool, and looked at platform-by-platform presence for searchinfluence.com over&#8230;<\/p>\n","protected":false},"author":1,"featured_media":725847,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/searchengineland.com\/wp-content\/seloads\/2026\/05\/Your-managed-WordPress-might-be-blocking-AI-bots-and-you-cant-see-it.png","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-725846","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/725846","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=725846"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/725846\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/725847"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=725846"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=725846"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=725846"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}