{"id":628671,"date":"2024-07-25T03:39:22","date_gmt":"2024-07-25T00:39:22","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/microsoft-confirms-reddit-blocked-bing-search\/"},"modified":"2024-07-25T03:39:22","modified_gmt":"2024-07-25T00:39:22","slug":"microsoft-confirms-reddit-blocked-bing-search","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/microsoft-confirms-reddit-blocked-bing-search\/","title":{"rendered":"#Microsoft confirms Reddit blocked Bing Search"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a271fa68ce7d\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a271fa68ce7d\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/microsoft-confirms-reddit-blocked-bing-search\/#Reddit_updated_its_robotstxt_file_prohibiting_Bing_and_some_other_search_engines_from_crawling_its_site\" >Reddit updated its robots.txt file prohibiting Bing and some other search engines from crawling its site.<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"subhead\" itemprop=\"alternativeHeadline\"><span class=\"ez-toc-section\" id=\"Reddit_updated_its_robotstxt_file_prohibiting_Bing_and_some_other_search_engines_from_crawling_its_site\"><\/span>Reddit updated its robots.txt file prohibiting Bing and some other search engines from crawling its site.<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><\/p>\n<div class=\"bialty-container\">\n<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">Reddit<\/a> has updated its robots.txt file, preventing Bing and many other search engines from crawling the site.  \u201cBing stopped crawling Reddit after they implemented their updated robots.txt file on July 1, which prohibits all crawling of their site,\u201d a Microsoft representative told Search Engine Land.<\/p>\n<p><strong>What h<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>ened. <\/strong>On July 1, 2024, Reddit updated its robots.txt file preventing many search engines and AI tools from crawling the site.  <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.seroundtable.com\/reddit-it-not-blocking-google-search-37671.html\">Reddit did not prevent Google<\/a> from crawling the site, despite what some people may have thought earlier this month.  But Reddit did prevent most other crawlers from crawling the site.<\/p>\n<p>Earlier this morning, Mark Williams-Cook <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/x.com\/thetafferboy\/status\/1816112568722731499\">notified<\/a> me that Reddit results were dropping out of the Bing Search index. Then <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.techmeme.com\/240724\/p29#a240724p29\">several media outlets<\/a> began covering the <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>.  I wanted to confirm that Bing\u2019s crawlers were indeed blocked, because Reddit was using IP detection to show search engines one version of its robots.txt file and humans another version of the robots.txt file \u2013 as I <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.seroundtable.com\/reddit-it-not-blocking-google-search-37671.html#:~:text=I%20mean%2C%20at%20first%20glance%2C%20it%20does%20look%20like%20Reddit%27s%20robots.txt%20file%20is%20blocking%20all%20search%20engines%2C%20including%20Google%20Search%3A\">explained<\/a> earlier this month.<\/p>\n<p>Bing has thus stopped crawling new content on Reddit, which is why when you filter Reddit results in Bing Search for the last week, <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.bing.com\/%2fsearch%3fq%3dsite%253areddit.com%26filters%3dex1%253a%2522ez5_19918_19928%2522%26FORM%3d000017%26sp%3d-1%26ghc%3d1%26lq%3d0%26pq%3dsite%253areddit.com%26sc%3d11-15%26qs%3dn%26cvid%3d8B90B38FB7E7439EB492A337ED92C0FD%26ghsh%3d0%26ghacc%3d0%26qpvt%3dsite%253areddit.com\">you see nothing<\/a>:<\/p>\n<figure class=\"wp-block-image size-full\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1744\" height=\"656\" alt=\"Bing Reddit Index Last Week\" class=\"wp-image-444386\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week.png.webp 1744w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-600x226.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-800x301.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-200x75.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-768x289.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-1536x578.png 1536w\" data-lazy-sizes=\"(max-width: 1744px) 100vw, 1744px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week.png.webp\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1744\" height=\"656\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week.png.webp\" alt=\"Bing Reddit Index Last Week\" class=\"wp-image-444386\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week.png.webp 1744w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-600x226.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-800x301.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-200x75.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-768x289.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/bing-reddit-index-last-week-1536x578.png 1536w\" sizes=\"(max-width: 1744px) 100vw, 1744px\"><\/figure>\n<p><strong>Microsoft confirmed. <\/strong>A Microsoft spokesperson told Search Engine Land:<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>\n\u201cWe respect the robots.txt standard. Bing stopped crawling Reddit after they implemented their updated robots.txt file on July 1, which prohibits all crawling of their site.\u201d\n<\/p><\/blockquote>\n<p><strong>Reddit statement. <\/strong>Reddit spokesperson Tim Rathschmidt says in a statement to The Verge:<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>\n\u201cThis is not at all related to our recent partnership with Google. We have been in discussions with multiple search engines. We have been unable to reach agreements with all of them, since some are unable or unwilling to make enforceable promises regarding their use of Reddit content, including their use for AI.\u201d\n<\/p><\/blockquote>\n<p><strong>Why we care. <\/strong>With Reddit securing a licensing deal with Google, Reddit is able to play hardball with other search engines and AI tools.  So Reddit has blocked most other search engines from crawling its content.  Meanwhile, Google is driving insane traffic to Reddit these days, including testing special treatment in its search results for them.<\/p>\n<p>It makes you wonder if other large websites can try to go down this route and where that might leave smaller publishers and content producers.<\/p>\n<p>Meanwhile, don\u2019t expect to see much new Reddit content being surfaced on Bing in the near future.\n<\/p><\/div>\n<p><\/p>\n<div class=\"about-author\">\n    About the author<\/p>\n<div class=\"information\">\n<div class=\"author-module\">\n<div class=\"row\">\n<div class=\"col-12 col-lg-3 text-center\">\n<div class=\"avatar\">\n                        <img loading=\"lazy\" decoding=\"async\" class=\"img-fluid rounded-circle avatar-border\" alt=\"Barry Schwartz\" width=\"140\" height=\"140\" src=\"https:\/\/searchengineland.com\/images\/authors\/BarrySchwartz-lg.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"img-fluid rounded-circle avatar-border\" src=\"https:\/\/searchengineland.com\/images\/authors\/BarrySchwartz-lg.jpg\" alt=\"Barry Schwartz\" width=\"140\" height=\"140\">\n                                            <\/div>\n<\/p><\/div>\n<div class=\"col-12 col-lg-9\">\n<div class=\"about\">\n<div class=\"name\">\n                            <strong>Barry Schwartz<\/strong>\n                        <\/div>\n<div class=\"row g-2 pt-2\">\n<div class=\"col-auto twitter\">\n                                    <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/intent\/follow?original_referer=https%3A%2F%2Fsearchengineland.com%2F&amp;region=follow_link&amp;screen_name=rustybrick&amp;tw_p=followbutton&amp;variant=2.0\" aria-label=\"opens in a new tab\"><i class=\"fab fa-x-twitter\"><\/i><\/a>\n                            <\/div>\n<div class=\"col-auto\">\n                                    <a rel=\"nofollow noopener\" target=\"_blank\" href=\"http:\/\/www.linkedin.com\/in\/rustybrick\" aria-label=\"opens in a new tab\"><i class=\"fab fa-linkedin\"><\/i><\/a>\n                                <\/div>\n<\/p><\/div>\n<p>                        Barry Schwartz is a technologist and a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.rustybrick.com\/\">RustyBrick<\/a>, a NY based web consulting firm. He also runs <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.seroundtable.com\/\">Search Engine Roundtable<\/a>, a popular search blog on very advanced SEM topics.<\/p>\n<p>In 2019, Barry was awarded the Outstanding Community Services Award from Search Engine Land, in 2018 he was awarded the US Search Awards the &#8220;US Search Personality Of The Year,&#8221; you can learn more over here and in 2023 he was listed as a top 50 most influential PPCer by Marketing O&#8217;Clock.<\/p>\n<p>Barry can be followed\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/rustybrick\/\">on X here<\/a>\u00a0and you can learn more about\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.rustybrick.com\/barry\">Barry Schwartz over here<\/a> or on her <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.barryschwartz.org\/\">personal site<\/a>.                 <\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology<\/a><\/span> category.<\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/searchengineland.com\/microsoft-confirms-reddit-blocked-bing-search-444385\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Reddit updated its robots.txt file prohibiting Bing and some other search engines from crawling its site. Reddit has updated its robots.txt file, preventing Bing and many other search engines from crawling the site. \u201cBing stopped crawling Reddit after they implemented their updated robots.txt file on July 1, which prohibits all crawling of their site,\u201d a&#8230;<\/p>\n","protected":false},"author":1,"featured_media":628672,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/searchengineland.com\/wp-content\/seloads\/2020\/03\/Reddit_1920-800x450.jpg","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[48041,148291,70286,78070],"class_list":["post-628671","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology","tag-bing","tag-bing-seo","tag-microsoft","tag-seo"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/628671","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=628671"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/628671\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/628672"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=628671"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=628671"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=628671"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}