{"id":656801,"date":"2025-03-11T22:10:16","date_gmt":"2025-03-11T19:10:16","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/ai-search-engines-often-make-up-citations-and-answers-study\/"},"modified":"2025-03-11T22:10:16","modified_gmt":"2025-03-11T19:10:16","slug":"ai-search-engines-often-make-up-citations-and-answers-study","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/ai-search-engines-often-make-up-citations-and-answers-study\/","title":{"rendered":"#AI search engines often make up citations and answers: Study"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a2811ed3668e\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a2811ed3668e\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/ai-search-engines-often-make-up-citations-and-answers-study\/#I_read_the_news_today_oh_boy_AI_search_engines_give_wrong_answers_60_of_the_time_and_often_cite_fabricated_or_broken_URLs\" >I read the news today, oh boy. AI search engines give wrong answers 60% of the time and often cite fabricated or broken URLs.<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"subhead\" itemprop=\"alternativeHeadline\"><span class=\"ez-toc-section\" id=\"I_read_the_news_today_oh_boy_AI_search_engines_give_wrong_answers_60_of_the_time_and_often_cite_fabricated_or_broken_URLs\"><\/span>I read the news today, oh boy. AI search engines give wrong answers 60% of the time and often cite fabricated or broken URLs.<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><\/p>\n<div class=\"bialty-container\">\n<p>AI search engines and chatbots often provide wrong answers and make up article citations, according to a new study from Columbia Journalism Review.<\/p>\n<p><strong>Why we care.<\/strong> AI search tools have ramped up the scraping of your content so they can serve answers to their users, often resulting in no clicks to your website. Also, click-through rates from AI search and chatbots are much lower than Google Search, according to a separate, unrelated study. But hallucinating citations makes an already bad situation even worse.<\/p>\n<p><strong>By the numbers.<\/strong> More than half of the responses from Gemini and Grok 3 cited fabricated or broken URLs\u00a0that led to error pages. Also, according to the study:<\/p>\n<ul class=\"wp-block-list\">\n<li>Overall, chatbots provided incorrect answers to more than 60% of queries:\n<ul class=\"wp-block-list\">\n<li>Grok 3 (the highest error rate) answered 94% of the queries incorrectly.<\/li>\n<li>Gemini only provided a completely correct response on one occasion (in 10 attempts).<\/li>\n<li>Perplexity, which had the lowest error rate, answered 37% of queries incorrectly.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>What they\u2019re saying.<\/strong> The study authors (Klaudia Ja\u017awi\u0144ska and Aisvarya Chandrasekar), who also noted that \u201cmultiple chatbots seemed to bypass Robot Exclusion Protocol preferences,\u201d summed up this way:<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u201cThe findings of this study align closely with those outlined in our previous\u00a0<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.cjr.org\/tow_center\/how-chatgpt-misrepresents-publisher-content.php\" target=\"_blank\" rel=\"noopener\">ChatGPT study<\/a>, published in November 2024, which revealed consistent patterns across chabots: confident presentations of incorrect information, misleading attributions to syndicated content, and inconsistent information retrieval practices. Critics of generative search like Chirag Shah and Emily M. Bender have raised substantive concerns about using large language models for search,\u00a0<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/3649468\" target=\"_blank\" rel=\"noopener\">noting<\/a>\u00a0that they \u2018take away transparency and user agency, further amplify the problems associated with bias in [information access] systems, and often provide ungrounded and\/or toxic answers that may go unchecked by a typical user.&#8217;\u201d\u00a0<\/p>\n<\/blockquote>\n<p><strong>About the comparison.<\/strong> This analysis of 1,600 queries compared the ability of generative AI tools (ChatGPT search, Perplexity, Perplexity Pro, DeepSeek search, Microsoft CoPilot, xAI\u2019s Grok-2 and Grok-3 search, and Google Gemini) to identify an article\u2019s headline, original publisher, publication date, and URL, based on direct excerpts of 10 articles chosen at random from 20 publishers.<\/p>\n<p><strong>The study.<\/strong> <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.cjr.org\/tow_center\/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php\" target=\"_blank\" rel=\"noopener\">AI Search Has A Citation Problem<\/a><\/p>\n<\/div>\n<p><\/p>\n<div class=\"about-author\">\n<p>About the author<\/p>\n<div class=\"information\">\n<div class=\"author-module\">\n<div class=\"row\">\n<div class=\"col-12 col-lg-3 text-center\">\n<div class=\"avatar\">\n\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" class=\"img-fluid rounded-circle avatar-border\" alt=\"Danny Goodwin\" width=\"140\" height=\"140\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/Danny-Goodwin-scaled.jpeg.webp\"><img loading=\"lazy\" decoding=\"async\" class=\"img-fluid rounded-circle avatar-border\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/07\/Danny-Goodwin-scaled.jpeg.webp\" alt=\"Danny Goodwin\" width=\"140\" height=\"140\">\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n<\/p><\/div>\n<div class=\"col-12 col-lg-9\">\n<div class=\"about\">\n<div class=\"name\">\n\t\t\t\t\t\t\t<strong>Danny Goodwin<\/strong>\n\t\t\t\t\t\t<\/div>\n<div class=\"row g-2 pt-2\">\n<div class=\"col-auto twitter\">\n\t\t\t\t\t\t\t\t\t<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/twitter.com\/intent\/follow?original_referer=https%3A%2F%2Fsearchengineland.com%2F&amp;region=follow_link&amp;screen_name=MrDannyGoodwin&amp;tw_p=followbutton&amp;variant=2.0\" rel=\"me\" target=\"_blank\" aria-label=\"opens in a new tab\"><i class=\"fab fa-x-twitter\"><\/i><\/a>\n\t\t\t\t\t\t\t<\/div>\n<div class=\"col-auto\">\n\t\t\t\t\t\t\t\t\t<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/in\/dannygoodwin\/\" target=\"_blank\" aria-label=\"opens in a new tab\"><i class=\"fab fa-linkedin\"><\/i><\/a>\n\t\t\t\t\t\t\t\t<\/div>\n<\/p><\/div>\n<p>\t\t\t\t\t\tDanny Goodwin is Editorial Director of Search Engine Land &amp; Search Marketing Expo &#8211; SMX. He joined Search Engine Land in 2022 as Senior Editor. In addition to reporting on the latest search marketing <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>, he manages Search Engine Land\u2019s SME (Subject Matter Expert) program. He also helps program U.S. SMX events. <\/p>\n<p>Goodwin has been editing and writing about the latest developments and trends in search and digital marketing since 2007. He previously was Executive Editor of Search Engine Journal (from 2017 to 2022), managing editor of Momentology (from 2014-2016) and editor of Search Engine Watch (from 2007 to 2014). He has spoken at many major search conferences and virtual events, and has been sourced for his expertise by a wide range of publications and podcasts.\t\t\t\t\t<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/category\/technology\/\" target=\"_blank\" >Technology<\/a><\/span> category.<\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/searchengineland.com\/ai-search-engines-citations-links-453173\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I read the news today, oh boy. AI search engines give wrong answers 60% of the time and often cite fabricated or broken URLs. AI search engines and chatbots often provide wrong answers and make up article citations, according to a new study from Columbia Journalism Review. Why we care. AI search tools have ramped&#8230;<\/p>\n","protected":false},"author":1,"featured_media":656802,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/searchengineland.com\/wp-content\/seloads\/2023\/07\/sad-robot-on-phone-1920.jpg","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[149959,78070],"class_list":["post-656801","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology","tag-news-publisher","tag-seo"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/656801","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=656801"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/656801\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/656802"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=656801"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=656801"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=656801"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}