{"id":684798,"date":"2025-08-13T07:30:29","date_gmt":"2025-08-13T04:30:29","guid":{"rendered":"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/"},"modified":"2025-08-13T07:30:29","modified_gmt":"2025-08-13T04:30:29","slug":"most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/","title":{"rendered":"Most SEO research doesn\u2019t lie \u2013 but doesn\u2019t tell the truth either"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a357ec383655\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a357ec383655\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#The_problems_with_unscientific_SEO_research\" >The problems with unscientific SEO research<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#AI_Overview_overlap_studies\" >AI Overview overlap studies<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#My_meta_study_AI_Overviews_vs_search_overlap\" >My meta study: AI Overviews vs. search overlap<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Profounds_%E2%80%98The_Surprising_Gap_Between_ChatGPT_and_Google\" >Profound\u2019s \u2018The Surprising Gap Between ChatGPT and Google\u2019<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#German_researchers_study_%E2%80%98Is_Google_Getting_Worse_and_multiple_surveys_on_the_same_question_from_Statista_The_Verge_and_Wallethub\" >German researchers\u2019 study, \u2018Is Google Getting Worse?,\u2019 and multiple surveys on the same question from Statista, The Verge, and Wallethub<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Adobes_%E2%80%98How_ChatGPT_is_changing_the_way_we_search\" >Adobe\u2019s \u2018How ChatGPT is changing the way we search\u2019<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Why_do_we_fall_victim_to_these_traps\" >Why do we fall victim to these traps?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Ignorance\" >Ignorance<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Stupidity\" >Stupidity&nbsp;<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Amathia_voluntary_stupidity\" >Amathia (voluntary stupidity)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Biases_emotions_hidden_agendas_and_incentives\" >Biases, emotions, hidden agendas, and incentives<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Reasons_why_we_fall_for_bad_research_in_SEO\" >Reasons why we fall for bad research in SEO<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#How_to_fix_all_of_this\" >How to fix all of this<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Conducting_SEO_research_with_scientific_rigor\" >Conducting SEO research with scientific rigor<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Reporting_more_accurately_on_SEO_research_%E2%80%93_and_news_in_general\" >Reporting more accurately on SEO research \u2013 and news in general<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Avoid_falling_for_bad_SEO_research\" >Avoid falling for bad SEO research<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Your_research_bulls_detector\" >Your research bulls*** detector<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/buradabiliyorum.com\/en\/most-seo-research-doesnt-lie-but-doesnt-tell-the-truth-either\/#Want_better_more_honest_SEO_research_Look_at_the_person_in_the_mirror\" >Want better, more honest SEO research? Look at the person in the mirror<\/a><\/li><\/ul><\/nav><\/div>\n<div><img loading=\"lazy\" decoding=\"async\" width=\"1920\" height=\"1080\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/Most-SEO-research-doesnt-lie-\u2013-but-doesnt-tell-the-truth-either.jpg\" class=\"attachment-large size-large wp-post-image\" alt=\"Most SEO research doesn\u2019t lie \u2013 but doesn\u2019t tell the truth either\" style=\"margin-bottom: 15px\" \/><\/div>\n<p>A mirage in the desert looks like water from a distance and can fool even experienced <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/trip-and-travel\/\" data-internallinksmanager029f6b8e52c=\"10\" title=\"Trip &amp; Travel\" target=\"_blank\" rel=\"noopener\">travel<\/a>ers into chasing something that isn\u2019t there.<\/p>\n<p>SEO research can be the same. <\/p>\n<p>It looks like <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/sciencee\/\" data-internallinksmanager029f6b8e52c=\"5\" title=\"Science\" target=\"_blank\" rel=\"noopener\">science<\/a>, sounds legitimate, and can trick even seasoned marketers into believing they\u2019ve found something real.<\/p>\n<p>Daniel Kahneman once said people would rather use a map of the Pyrenees while lost in the Alps than have no map at all. <\/p>\n<p>In SEO, we take it further: we use a map of the Pyrenees, call it the Alps, and then confidently teach others our \u201cnavigation techniques.\u201d<\/p>\n<p>Worse still, most of us rarely question the authorities presenting these maps.<\/p>\n<p>As Albert Einstein said, \u201cBlind obedience to authority is the greatest enemy of the truth.\u201d&nbsp;<\/p>\n<p>It\u2019s time to stop chasing mirages and start demanding better maps.<\/p>\n<p>This article shows:<\/p>\n<ul class=\"wp-block-list\">\n<li>How unscientific SEO research misleads us.<\/li>\n<li>Why we keep falling for it.<\/li>\n<li>What we can do to change that.<\/li>\n<\/ul>\n<p><strong>Spoiler<\/strong>: I\u2019ll also share a prompt I created to quickly spot pseudoscientific SEO studies \u2013 so you can avoid bad decisions and wasted time.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-the-problems-with-unscientific-seo-research\"><span class=\"ez-toc-section\" id=\"The_problems_with_unscientific_SEO_research\"><\/span>The problems with unscientific SEO research<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Real research should map the terrain and either validate or falsify your techniques.&nbsp;<\/p>\n<p>It should show:<\/p>\n<ul class=\"wp-block-list\">\n<li>Which routes lead to the summit and which end in deadly falls.<\/li>\n<li>What gear will actually hold under pressure.<\/li>\n<li>Where the solid handholds are \u2013 versus the loose rock that crumbles when you need it most.<\/li>\n<\/ul>\n<p>Bad research sabotages all of that. Instead of standing on solid ground, you\u2019re balancing on a shaky foundation.<\/p>\n<p>Take one common example: \u201cWe GEO\u2019d our clients to X% more traffic from ChatGPT.\u201d&nbsp;<\/p>\n<p>These studies often skip a critical factor \u2013 ChatGPT\u2019s own natural growth.&nbsp;<\/p>\n<p>Between September 2024 and July 2025, chatgpt.com\u2019s traffic jumped from roughly 3 billion visits to 5.5 billion \u2013 an 83% increase.&nbsp;<\/p>\n<p>That growth alone could explain the numbers.<\/p>\n<p>Yet these findings are repackaged into sensational headlines that flood <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">social media<\/a>, boosted by authoritative accounts with massive followings.<\/p>\n<p>Most of these studies fail the basics.&nbsp;<\/p>\n<p>They lack replicability and can\u2019t be <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a>ized.<\/p>\n<p>Yet they are presented as if they are the definitive map for navigating the foggy AI mountain we\u2019re climbing.<\/p>\n<p>Let\u2019s look at some examples of dubious SEO research.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-ai-overview-overlap-studies\"><span class=\"ez-toc-section\" id=\"AI_Overview_overlap_studies\"><\/span>AI Overview overlap studies<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>AI Overview overlap studies try to explain how much influence traditional SEO rankings have on <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>earing inside AI Overviews \u2013 often considered the new peak in organic search.&nbsp;<\/p>\n<p>Since its original inception as Search Generative Experience (SGE), dozens of these overlap studies have emerged.<\/p>\n<p>I\u2019ve read through all of them \u2013 so you don\u2019t have to \u2013 and pulled together my own non-scientific meta study.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-my-meta-study-ai-overviews-vs-search-overlap\"><span class=\"ez-toc-section\" id=\"My_meta_study_AI_Overviews_vs_search_overlap\"><\/span>My meta study: AI Overviews vs. search overlap<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>I went back to early 2024, reviewed every study I could find, and narrowed them down to 11 that met three basic criteria:<\/p>\n<ul class=\"wp-block-list\">\n<li>Comparison of URLs, not domains.<\/li>\n<li>Measure the overlap of the organic Top 10 with the AI Overviews URLs.<\/li>\n<li>Based on all URLs in the Top 10, not just 1.<\/li>\n<\/ul>\n<p>The end result (sorted by overlap in %):<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"1600\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/Meta-study-AI-Overviews-vs.-search-overlap.png\" alt=\"Meta study- AI Overviews vs. search overlap\" class=\"wp-image-460606\" \/><\/figure>\n<\/div>\n<ul class=\"wp-block-list\">\n<li>Overlap ranged from 5-77%<\/li>\n<li>Average: 45.84%<\/li>\n<li>Median: 46.40%<\/li>\n<\/ul>\n<p>These huge discrepancies come down to a few factors:<\/p>\n<ul class=\"wp-block-list\">\n<li>Different numbers of keywords.<\/li>\n<li>Different keyword sets in general.<\/li>\n<li>Different time frames.<\/li>\n<li>Likely different keyword types.<\/li>\n<\/ul>\n<p>In summary:<\/p>\n<ul class=\"wp-block-list\">\n<li>Most studies focused on the U.S. market.&nbsp;<\/li>\n<li>Only one provided a dataset for potential peer review.&nbsp;<\/li>\n<li>Just two included more than 100,000 keywords.<\/li>\n<li>And none explained in detail how the keywords were chosen.<\/li>\n<\/ul>\n<p>There are only two noteworthy patterns across the studies:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Over time, inclusion in the organic Top 10 seems to make it more likely to rank in AI Overviews<\/strong>.<\/li>\n<\/ul>\n<p>In other words, Google now seems to rely more heavily on Top 10 results for AI Overview content than it did in the early days.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1201\" height=\"901\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/A-chart-of-the-AIO-overlap-data-shown-over-time.png\" alt=\"A chart of the AIO overlap data shown over time. It gradually increases, has a strong peak and a strong decline in 2 spots that are marked in red.\" class=\"wp-image-460608\" \/><\/figure>\n<\/div>\n<p>If we exclude these studies (marked in the graph above) that didn\u2019t disclose the number of keywords, we get this graph:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1201\" height=\"901\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/A-chart-with-the-headline-22Over-time-inclusion-is-more-likely-if-you-rank-in-the-top-1022.png\" alt=\"A chart with the headline &quot;Over time, inclusion is more likely if you rank in the top 10. The graph once again shows AIO overlap in % over time. While there are some peaks, the generel trend is upwards.\" class=\"wp-image-460609\" \/><\/figure>\n<\/div>\n<ul class=\"wp-block-list\">\n<li><strong>Ranking in the Top 10 correlates with being more likely to also rank in an AI Overviews<\/strong>.<\/li>\n<\/ul>\n<p>That\u2019s it.&nbsp;But even then, there are several reasons why these studies are generally flawed.<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>None of the studies uses a keyword set big enough:<\/strong> The results cannot be generalized, like mapping one cliff face and claiming it applies to the entire mountain range.<\/li>\n<li><a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/posts\/joshua-blyskal_ai-search-volatility-is-far-greater-than-activity-7351265339621486593-AzGP\/\" target=\"_blank\" rel=\"noopener\"><strong>AI is always changing<\/strong><\/a><strong> \u2013 <\/strong><strong>and always has been<\/strong><strong>:<\/strong> The insights become outdated quickly, like GPS directions to a road that no longer exists.<\/li>\n<li><strong>It\u2019s not always clear what was measured:<\/strong> Some reports are <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/posts\/marcustober_still-think-google-ai-mode-is-the-same-as-ugcPost-7348337146392403968-9LXU\/\" target=\"_blank\" rel=\"noopener\">promoted with obscure marketing material<\/a>, and you wouldn\u2019t understand them without the additional context \u2013 like a gear review that never mentions what type of rock it was tested on.<\/li>\n<li><strong>Too much focus on averages \u2013 and averages are dangerous:<\/strong> For one keyword type or niche, the overlap might be low. For others, it might be high. The average is in the middle. It&#8217;s like a bridge built for average traffic \u2013 handles normal loads fine, but collapses when the heavy trucks come.<\/li>\n<li><strong>Ignore query fan-out in the analysis:<\/strong> These studies give directions for where to go \u2013 too bad they\u2019re driving a car while we\u2019re in a boat. All major AI chatbots use query fan-out, yet none of the studies accounted for it.&nbsp;<\/li>\n<\/ul>\n<p>This isn\u2019t new knowledge. Google filed a <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/patents.google.com\/patent\/US11769017B1\/en\" target=\"_blank\" rel=\"noopener\">patent for generative engine summaries<\/a> in March 2023, stating that they also use search result documents (SRDs) that are:<\/p>\n<ul class=\"wp-block-list\">\n<li>Related-query-responsive.<\/li>\n<li>Recent-query-responsive.<\/li>\n<li>Implied-query-responsive.<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1201\" height=\"1310\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/Google-patent-on-generative-summaries.png\" alt=\"Google's patent on generative summaries for search results. Figure 2 is shown and related-query-responsive SRD(s), recent-query-responsive SRD(s), and implied-query-responsive SRD(s) are highlighted.\" class=\"wp-image-460607\" \/><\/figure>\n<\/div>\n<p>Google may not have marketed this until May 2025, but it\u2019s been in plain sight for over two years.<\/p>\n<p>The real overlap of AI Overviews with Google Search depends on the overlap of all queries used, including synthetic queries.&nbsp;<\/p>\n<p>If you can\u2019t measure that, at least mention it as part of your limitations going forward.<\/p>\n<p>Here are three more examples of recent SEO research that I find questionable.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-profound-s-the-surprising-gap-between-chatgpt-and-google\"><span class=\"ez-toc-section\" id=\"Profounds_%E2%80%98The_Surprising_Gap_Between_ChatGPT_and_Google\"><\/span>Profound\u2019s \u2018<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.tryprofound.com\/guides\/the-surprising-gap-between-chatgpt-and-google\" target=\"_blank\" rel=\"noopener\">The Surprising Gap Between ChatGPT and Google<\/a>\u2019<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Marketed as \u201cwow, only 8-12% overlap between ChatGPT and Google Search Top 10 results,\u201d this claim is actually based on just two queries repeated a few hundred times.&nbsp;<\/p>\n<p>I seriously doubt the data provider considered this high-quality research.&nbsp;<\/p>\n<p>Yet, despite its flaws, it\u2019s been widely shared by creators.<\/p>\n<p><figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"440\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/06\/semrush-discover-ai-optimization.png\" alt=\"\" class=\"wp-image-458321\" \/><\/figure>\n<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-german-researchers-study-is-google-getting-worse-and-multiple-surveys-on-the-same-question-from-statista-the-verge-and-wallethub\"><span class=\"ez-toc-section\" id=\"German_researchers_study_%E2%80%98Is_Google_Getting_Worse_and_multiple_surveys_on_the_same_question_from_Statista_The_Verge_and_Wallethub\"><\/span>German researchers\u2019 study, \u2018<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/downloads.webis.de\/publications\/papers\/bevendorff_2024a.pdf\" target=\"_blank\" rel=\"noopener\">Is Google Getting Worse?<\/a>,\u2019 and multiple surveys on the same question from <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.statista.com\/statistics\/185966\/us-customer-satisfaction-with-google\/\" target=\"_blank\" rel=\"noopener\">Statista<\/a>, <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.theverge.com\/press-room\/617654\/internet-community-future-research\" target=\"_blank\" rel=\"noopener\">The Verge<\/a>, and <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/wallethub.com\/edu\/google-search-results-study\/139920#survey\" target=\"_blank\" rel=\"noopener\">Wallethub<\/a><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>I covered these in my article, \u201cIs Google really getting worse? (Actually, it\u2019s complicated).\u201d&nbsp;<\/p>\n<p>In short, the study has been frequently misquoted.<\/p>\n<p>The surveys:<\/p>\n<ul class=\"wp-block-list\">\n<li>Contradict one another.<\/li>\n<li>Often use suggestive framing.<\/li>\n<li>Rely on what people say rather than what they actually do.<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\" id=\"h-adobe-s-how-chatgpt-is-changing-the-way-we-search\"><span class=\"ez-toc-section\" id=\"Adobes_%E2%80%98How_ChatGPT_is_changing_the_way_we_search\"><\/span>Adobe\u2019s \u2018<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.adobe.com\/express\/learn\/blog\/chatgpt-as-a-search-engine\" target=\"_blank\" rel=\"noopener\">How ChatGPT is changing the way we search<\/a>\u2019<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>A survey with only 1,000 people participating, 200 of them being marketers and small business owners \u2013 all of them using ChatGPT.<\/p>\n<p>Yet, they promote the survey, stating that \u201c77% of people in the U.S. use ChatGPT as a search engine.\u201d<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-why-do-we-fall-victim-to-these-traps\"><span class=\"ez-toc-section\" id=\"Why_do_we_fall_victim_to_these_traps\"><\/span>Why do we fall victim to these traps?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Not all SEO research is unscientific for the same reasons. I see four main causes.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-ignorance\"><span class=\"ez-toc-section\" id=\"Ignorance\"><\/span>Ignorance<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Ignorance is like darkness.&nbsp;<\/p>\n<p>At nighttime, it\u2019s natural to have an impoverished sight.&nbsp;<\/p>\n<p>It means \u201cI don\u2019t know better (yet).\u201d&nbsp;<\/p>\n<p>You are currently missing the capability and knowledge to conduct scientific research. It\u2019s more or less neutral.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-stupidity-nbsp\"><span class=\"ez-toc-section\" id=\"Stupidity\"><\/span>Stupidity&nbsp;<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>This is when you are literally incapable, therefore also neutral. You just can\u2019t.&nbsp;<\/p>\n<p>Few people are intellectually capable of working in a position to conduct research and then fail to do so.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-amathia-voluntary-stupidity\"><span class=\"ez-toc-section\" id=\"Amathia_voluntary_stupidity\"><\/span>Amathia (voluntary stupidity)<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Worse than both is when the lights are on and you still decide not to see. Then you don\u2019t lack knowledge, but deny it.&nbsp;<\/p>\n<p>This is described as \u201cAmathia\u201d in Greek. You could know better, but actively seek out not to.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1201\" height=\"901\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/Amathia-is-voluntary-stupidity.png\" alt=\"A pyramid with the headline &quot;Amathia is voluntary stupidity&quot;. At the bottom there is stupidity. Then there is ignorance. On top is Amathia. Over time, danger increases.\" class=\"wp-image-460610\" \/><\/figure>\n<\/div>\n<p>While all forms are dangerous, Amathia is the most dangerous.&nbsp;<\/p>\n<p>Amathia resists correction, insists it is good, and actively misleads others.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-biases-emotions-hidden-agendas-and-incentives\"><span class=\"ez-toc-section\" id=\"Biases_emotions_hidden_agendas_and_incentives\"><\/span>Biases, emotions, hidden agendas, and incentives<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>You want to be right and can\u2019t see clearly, or openly try to deceive others.<\/p>\n<p>You don\u2019t have to lie to not tell the truth. You can deceive yourself just as well as you can deceive others.&nbsp;<\/p>\n<p>The best way to convince others of something is if you actually believe it yourself. We are masters at self-deception.<\/p>\n<p>Few promote products\/services they don\u2019t believe in themselves.&nbsp;<\/p>\n<p>You just don\u2019t realize the tricks a paycheck plays on your perception of reality.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-reasons-why-we-fall-for-bad-research-in-seo\"><span class=\"ez-toc-section\" id=\"Reasons_why_we_fall_for_bad_research_in_SEO\"><\/span>Reasons why we fall for bad research in SEO<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>We have the ability to open our minds more than ever before.&nbsp;<\/p>\n<p>Yet, we decide to shrink ourselves down.<\/p>\n<p>This is encouraged in part because of smartphones and social media, both induced by big tech companies, which are also responsible for the greatest theft of mankind (you could call it Grand Theft AI or GTAI).<\/p>\n<p>In a 2017 interview, <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.axios.com\/2017\/12\/15\/sean-parker-unloads-on-facebook-god-only-knows-what-its-doing-to-our-childrens-brains-1513306792\" target=\"_blank\" rel=\"noopener\">Facebook\u2019s founding president Sean Parker said<\/a>:<\/p>\n<ul class=\"wp-block-list\">\n<li>&#8220;The thought process that went into building these applications, Facebook being the first of them, &#8230; was all about: &#8216;How do we consume as much of your time and conscious attention as possible?&#8217; And that means that we need to sort of give you a little dopamine hit every once in a while. [&#8230;] It&#8217;s a social-validation feedback loop &#8230; exactly the kind of thing that a hacker like myself would come up with, because you&#8217;re exploiting a vulnerability in human psychology.&#8221;<\/li>\n<\/ul>\n<p>They don\u2019t care what kind of engagement they get. Fake <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a> that polarizes?&nbsp;Great, give it a boost.&nbsp;<\/p>\n<p>Most people are stuck in this hamster wheel of being bombarded with crap all day.&nbsp;<\/p>\n<p>The only missing piece? A middleman that amplifies. Those are content creators, publishers, news outlets, etc.<\/p>\n<p>Now we have a loop.&nbsp;<\/p>\n<ul class=\"wp-block-list\">\n<li>Platforms where research providers publish questionable studies.<\/li>\n<li>Amplifiers seeking engagement for personal gain.<\/li>\n<li>Consumers overwhelmed by a flood of information are all flooded with data.<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1201\" height=\"901\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/The-loop-of-doom.png\" alt=\"The loop of doom. Social media platforms are a foundation for research providers and amplifiers. Lastly, there is consumers. They all meet in a roundabout.\" class=\"wp-image-460611\" \/><\/figure>\n<\/div>\n<p>We are stuck in social media echo chambers.&nbsp;<\/p>\n<p>We want simple answers, and we are mostly driven by our emotions.&nbsp;<\/p>\n<p>And social media plays into all of that.<\/p>\n<p><!-- START INLINE FORM --><\/p>\n<div class=\"nl-inline-form border py-2 px-1 my-2\">\n<div class=\"row align-items-center nl-inline-container\">\n<div class=\"col-12 col-lg-3 col-xl-4 pe-md-0 pb-2 pb-lg-0\">\n<p class=\"inline-form-text text-center mb-0\">Get the newsletter search marketers rely on.<\/p>\n<\/p><\/div>\n<div class=\"col-12 col-lg-7 col-xl-6  pe-lg-0\">\n<div class=\"form-nl-inline\">\n<p>\t\t\t\t\tMktoForms2.loadForm(&#8220;https:\/\/app-sj02.marketo.com&#8221;, &#8220;727-ZQE-044&#8221;, 16298, function(form) {<br \/>\n\t\t\t\t\t\t\/\/ form.onSubmit(function(){<br \/>\n\t\t\t\t\t\t\/\/ });<\/p>\n<p>\t\t\t\t\t\t\/\/ form.onSuccess(function (values, followUpUrl) {<br \/>\n\t\t\t\t\t\t\/\/ });<br \/>\n\t\t\t\t\t});<\/p><\/div>\n<\/p><\/div>\n<div class=\"col-12 col-lg-2 col-xl-2 ps-1\">\n<p class=\"text-center mb-0\" style=\"font-size: .75rem\">See terms.<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n<p><!-- END INLINE FORM --><\/p>\n<hr class=\"wp-block-separator has-text-color has-cyan-bluish-gray-color has-css-opacity has-cyan-bluish-gray-background-color has-background\" \/>\n<h2 class=\"wp-block-heading\" id=\"h-how-to-fix-all-of-this\"><span class=\"ez-toc-section\" id=\"How_to_fix_all_of_this\"><\/span>How to fix all of this<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>As outlined throughout, we have three points that need fixing.&nbsp;<\/p>\n<ul class=\"wp-block-list\">\n<li>Conducting the research.<\/li>\n<li>Reporting on the research.<\/li>\n<li>Consuming the research.&nbsp;<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\" id=\"h-conducting-seo-research-with-scientific-rigor\"><span class=\"ez-toc-section\" id=\"Conducting_SEO_research_with_scientific_rigor\"><\/span>Conducting SEO research with scientific rigor<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Philosopher Karl Popper said that what scientists do is to try and prove they were wrong in what they do or believe.&nbsp;<\/p>\n<p>Most of us move the other way, trying to prove we\u2019re right. This is a mindset problem.&nbsp;<\/p>\n<p>Research is more convincing when you try to prove yourself wrong.<\/p>\n<p>Think steelmanning &gt; strawmanning.&nbsp;<\/p>\n<p>Ask yourself if the opposite of what you believe could also be true, and seek out data and arguments.&nbsp;<\/p>\n<p>You sometimes also have to accept the fact that you can be wrong or not have an answer.<\/p>\n<p>A few other things that would improve most SEO research:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Peer reviews<\/strong>: Provide the dataset you used and let others verify your findings. That automatically increases the believability of your work.<\/li>\n<li><strong>Observable behavior<\/strong>: Focus less on what is said and more on what you can see. What people say is almost never what they truly feel, believe, or do.<\/li>\n<li><strong>Continuous observation<\/strong>: Search quality and AI vs. search overlap are constantly changing, so they should also be observed and studied continuously.<\/li>\n<li><strong>Rock-solid study design<\/strong>: Read a good book on how to do scientific research. (Consider the classic, &#8220;<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/press.uchicago.edu\/ucp\/books\/book\/chicago\/C\/bo215874008.html\" target=\"_blank\" rel=\"noopener\">The Craft of Research<\/a>.&#8221;) Implement aspects like having test and control groups, randomization, acknowledging limitations, etc.<\/li>\n<\/ul>\n<p>I know that we can do better.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-reporting-more-accurately-on-seo-research-and-news-in-general\"><span class=\"ez-toc-section\" id=\"Reporting_more_accurately_on_SEO_research_%E2%80%93_and_news_in_general\"><\/span>Reporting more accurately on SEO research \u2013 and news in general<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Controversial and questionable studies gain traction through attention and a lack of critical thinking.<\/p>\n<p>Responsibility lies not just with the \u201cresearchers\u201d but also with those who amplify their work.<\/p>\n<p>What might help bring more balance to the conversation?<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Avoid sensationalism<\/strong>: It\u2019s likely that 80% of people only read the headline, so while it has to be click-attractive, it should avoid being click-baity.<\/li>\n<li><strong>Read yourself<\/strong>: Don\u2019t be a parrot of what other people say. Be very careful with AI summaries. Remember:\n<ul class=\"wp-block-list\">\n<li><a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/huggingface.co\/blog\/davidberenstein1957\/phare-analysis-of-hallucination-in-leading-llms\" target=\"_blank\" rel=\"noopener\">Brief summaries are more likely to include hallucinations<\/a>.<\/li>\n<li><a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/asia.nikkei.com\/Business\/Technology\/Artificial-intelligence\/Positive-review-only-Researchers-hide-AI-prompts-in-papers?utm_campaign=GL_asia_daily&amp;utm_medium=email&amp;utm_source=NA_newsletter&amp;utm_content=article_link&amp;del_type=1&amp;pub_date=20250701123000&amp;seq_num=18&amp;si=dc13b238-7871-468b-9eb7-69ba55978120\" target=\"_blank\" rel=\"noopener\">Prompt injection can manipulate the output<\/a>.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Check the (primary) sources<\/strong>: Whether it\u2019s an AI chatbot or someone else reporting on something, always check sources.<\/li>\n<li><strong>Have a critical stance<\/strong>: There is naive optimism and informed skepticism. Always ask yourself, \u201cDoes this make sense?\u201d<\/li>\n<\/ul>\n<p>Value truth over being first. That\u2019s journalism\u2019s responsibility.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-avoid-falling-for-bad-seo-research\"><span class=\"ez-toc-section\" id=\"Avoid_falling_for_bad_SEO_research\"><\/span>Avoid falling for bad SEO research<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>A curious mind is your best friend.&nbsp;<\/p>\n<p>Socrates used to ask a lot of questions to expose gaps in people\u2019s knowledge.&nbsp;<\/p>\n<p>Using this method, you can uncover whether the researchers have solid evidence for their claims or if they are drawing conclusions that their data doesn\u2019t actually support.<\/p>\n<p>Here are some questions that are worth asking:<\/p>\n<ul class=\"wp-block-list\">\n<li>Who conducted the research?\n<ul class=\"wp-block-list\">\n<li>Who are the people behind it?<\/li>\n<li>What is their goal?<\/li>\n<li>Are there any conflicts of interest?<\/li>\n<li>What incentives could influence their judgment?<\/li>\n<\/ul>\n<\/li>\n<li>How solid is the methodology of the study?\n<ul class=\"wp-block-list\">\n<li>What time frame was used for the study?<\/li>\n<li>Did they have test and control groups and were they observing or surveying?<\/li>\n<li>Under what criteria was the sample selected?<\/li>\n<li>Are the results statistically significant?<\/li>\n<\/ul>\n<\/li>\n<li>How generalizable and replicable are the results?\n<ul class=\"wp-block-list\">\n<li>Did they differentiate between geolocations?<\/li>\n<li>How big was the sample size?<\/li>\n<li>Do they talk about replicability and potential peer reviews?<\/li>\n<li>In what way are they talking about limitations of their research?<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>It\u2019s unlikely that you can ask too many questions and will end up drinking hemlock like Socrates.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-your-research-bulls-detector\"><span class=\"ez-toc-section\" id=\"Your_research_bulls_detector\"><\/span>Your research bulls*** detector<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>To leave you with something actionable, I built a prompt that you can use to assess research.<\/p>\n<p>Copy the following prompt:<\/p>\n<pre class=\"wp-block-code\"><code># Enhanced Research Evaluation Tool\n\nYou are a *critical research analyst. Your task is to evaluate a research article, study, experiment, or survey based on **methodological integrity, clarity, transparency, bias, reliability, and **temporal relevance*.\n\n---\n\n## Guiding Principles\n\n- Always *flag missing or unclear information*.\n- Use *explicit comments* for *anything ambiguous* that requires manual follow-up.\n- Don't add emojis to headlines unless provided in the prompt.\n- Apply *domain-aware scrutiny* to *timeliness. In rapidly evolving fields (e.g., AI, genomics, quantum computing), data, tools, or models older than **12\u201318 months* may already be outdated. In slower-moving disciplines (e.g., historical linguistics, geology), older data may still be valid.\n- Use your own corpus knowledge to assess what counts as *outdated*, and if uncertain, flag the timeframe as needing expert verification.\n- &#x1f4c8; All scores use the same logic:  \n  \u27a4 *Higher = better*  \n  \u27a4 For bias and transparency, *higher = more transparent and reliable*  \n  \u27a4 For evidence and methodology, *higher = more rigorous and valid*\n\n- *AI-specific guidance*:  \n   - Use of *GPT-3.5 or earlier (e.g., GPT-3.5 Turbo, DaVinci-003)* after 2024 should be treated as *outdated unless explicitly justified*.  \n   - Models such as *GPT-4o, Claude 4, Gemini 2.5* are considered current *as of mid 2025*.  \n   - *Flag legacy model use* unless its relevance is argued convincingly.\n\n---\n\n## 1. Extract Key Claims and Evidence\n\n| *Claim* | *Evidence Provided* | *Quote\/Passage* | *Supported by Data?* | *Score (1\u20136)* | *Emoji* | *Comment* |\n|----------|------------------------|--------------------|-------------------------|------------------|-----------|-------------|\n|          |                        |                    | Yes \/ No \/ Unclear      |                  | &#x1f7e5;&#x1f7e7;&#x1f7e9;       | Explain rationale. Flag ambiguous or unsupported claims. |\n\n*Legend* (for Claims &amp; Evidence Strength):  \n&#x1f7e5; = Weak (1\u20132)\u2003&#x1f7e7; = Moderate (3\u20134)\u2003&#x1f7e9; = Strong (5\u20136)\u2003Unclear = Not Provided or Needs Review  \n&#x1f4c8; Higher score = better support and stronger evidence\n\n---\n\n## 2. Evaluate Research Design and Methodology\n\n| *Criteria* | *Score (1\u20136)* | *Emoji* | *Comment \/ Flag* |\n|--------------|------------------|-----------|---------------------|\n| Clarity of hypothesis or thesis                        |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Sample size adequacy                                    |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Sample selection transparency (e.g., age, location, randomization) | | &#x1f7e5;&#x1f7e7;&#x1f7e9; |         |\n| Presence of test\/control groups (or clarity on observational methods) | | &#x1f7e5;&#x1f7e7;&#x1f7e9; |       |\n| *Time frame of the study (data collection window)*    | ? \/ 1\u20136 | Unclear \/ &#x1f7e5;&#x1f7e7;&#x1f7e9; | If not disclosed, mark as Unclear. If disclosed, assess whether the data is still timely for the domain. |\n| *Temporal Relevance* (Is the data or model still valid?) | ? \/ 1\u20136 | Unclear \/ &#x1f7e5;&#x1f7e7;&#x1f7e9; | Use domain-aware judgment. For example:  \n   - AI\/biotech = &lt; 12 months preferred  \n   - Clinical = within 3\u20135 years  \n   - History\/philosophy = lenient  \n   - For AI, if models like *GPT-3.5 or earlier* are used without explanation, flag as outdated. |\n| Data collection methods described                       |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Statistical testing \/ significance explained            |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Acknowledgment of limitations                           |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Provision of underlying data \/ replicability info       |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Framing and neutrality (no sensationalism or suggestive language) | | &#x1f7e5;&#x1f7e7;&#x1f7e9; |   |\n| Bias minimization (e.g., blinding, naturalistic observation) |      | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Transparency about research team, funders, affiliations |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Skepticism vs. naive optimism                           |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n\n*Legend* (for Methodology):  \n&#x1f7e5; = Poor (1\u20132)\u2003&#x1f7e7; = Moderate (3\u20134)\u2003&#x1f7e9; = Good (5\u20136)\u2003Unclear = Not Specified \/ Requires Manual Review  \n&#x1f4c8; Higher score = better design and methodological clarity\n\n---\n\n## 3. Bias Evaluation Tool\n\n| *Bias Type* | *Score (1\u20136)* | *Emoji* | *Comment* |\n|---------------|------------------|-----------|-------------|\n| Political Bias or Framing            |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Economic\/Corporate Incentives       |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Ideological\/Advocacy Bias           |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n| Methodological Bias (design favors specific outcome) | | &#x1f7e5;&#x1f7e7;&#x1f7e9; |     |\n| Lack of Disclosure or Transparency  |          | &#x1f7e5;&#x1f7e7;&#x1f7e9; |             |\n\n*Legend* (for Bias):  \n&#x1f7e5; = Low transparency (1\u20132)\u2003&#x1f7e7; = Moderate (3\u20134)\u2003&#x1f7e9; = High transparency (5\u20136)  \n&#x1f4c8; Higher score = less bias, more disclosure\n\n---\n\n## 4. Summary Box\n\n### Scores\n\n| *Category*                  | *Summary* |\n|------------------------------|-------------|\n| *Average Methodology Score* | X.X \/ 6 &#x1f7e5;&#x1f7e7;&#x1f7e9; (higher = better) |\n| *Average Bias Score*        | X.X \/ 6 &#x1f7e5;&#x1f7e7;&#x1f7e9; (higher = better transparency and neutrality) |\n| *Judgment*        | &#x2705; Trustworthy \/ &#x26a0; Needs Caution \/ &#x274c; Unreliable |\n| *Comment*  | e.g., \u201cStudy relies on outdated models (GPT-3.5),\u201d \u201cTime window not disclosed,\u201d \u201cHighly domain-specific assumptions\u201d |\n\n---\n\n### &#x1f44d; Strengths\n- ...\n- ...\n- ...\n\n### &#x1f44e; Weaknesses\n- ...\n- ...\n- ...\n\n### &#x1f6a9; Flag \/ Warnings\n- ...\n- ...\n- ...<\/code><\/pre>\n<\/p>\n<p>Here\u2019s an example output of the study on <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/2311.09735\" target=\"_blank\" rel=\"noopener\">generative engine optimization<\/a>:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1162\" height=\"1600\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/An-example-output-of-the-prompt-cited-above.png\" alt=\"An example output of the prompt cited above. There are number 1 and 2. 1 shows the claims made including columns like evidence provided, quote\/passage, a score, and a comment. 2 is an evaluation of the research design and methodology, following a similar column layout including scores.\" class=\"wp-image-460612\" \/><\/figure>\n<\/div>\n<ul class=\"wp-block-list\">\n<li>What claims are made and how they are supported.<\/li>\n<li>How the research design and methodology fare.<\/li>\n<\/ul>\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh7-rt.googleusercontent.com\/docsz\/AD_4nXfF0XRcT0x-xnujCXwKRSrpq6-VdYE0TJ7z8rXWC6xBbxU9xojMBflvC6kPc04LdYkSWK_vyud_qOaMw3vt4uQKCigmdPL7hW49HTOgXAgb_q8OJKILlgtzr_Q4nOT2F313JAOVgA?key=sOgke9E9rMwO06S-Jnr88g\" alt=\"3 and 4 show the bias evaluation, once again with scores. Lastly, 4 shows a summary of the criteria, including strengths, weaknesses, and flags\/warnings.\" \/><\/figure>\n<ul class=\"wp-block-list\">\n<li>Potential biases that are visible in the research.<\/li>\n<li>A summary box with strengths, weaknesses, and potential flags\/warnings.<\/li>\n<\/ul>\n<p>This study scores high as it follows a robust scientific methodology. The researchers even provided their dataset. (<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/generative-engines.com\/GEO\/\" target=\"_blank\" rel=\"noopener\">I checked the link<\/a>.)&nbsp;<\/p>\n<p>Important notes:&nbsp;<\/p>\n<ul class=\"wp-block-list\">\n<li>An analysis like this doesn\u2019t replace taking a look yourself or thinking critically about the information presented. What it can do, however, is to give you an indication if what you\u2019re reading is inherently flawed.<\/li>\n<li>If the researchers include some form of prompt injection that is supposed to manipulate an evaluation, you could get a wrong evaluation.<\/li>\n<\/ul>\n<p>That said, working with a structured prompt like this will yield much better results than \u201csummarize this study briefly.\u201d<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-want-better-more-honest-seo-research-look-at-the-person-in-the-mirror\"><span class=\"ez-toc-section\" id=\"Want_better_more_honest_SEO_research_Look_at_the_person_in_the_mirror\"><\/span>Want better, more honest SEO research? Look at the person in the mirror<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>SEO is not deterministic \u2013 it\u2019s not predictable with a clear cause-and-effect relationship.<\/p>\n<p>Most of what we do in SEO is probabilistic.&nbsp;<\/p>\n<p>Uncertainty and randomness always play a part, even though we often don\u2019t like to admit it.<\/p>\n<p>As a result, SEO research can\u2019t and doesn\u2019t have to meet other disciplines\u2019 standards.&nbsp;<\/p>\n<p>But the uncomfortable truth is that our industry&#8217;s hunger for certainty has created a marketplace for false confidence.&nbsp;<\/p>\n<p>We&#8217;ve built an ecosystem where suspect research gets rewarded with clicks and authority while rigorous honesty gets ignored, left alone in the dark.<\/p>\n<p>The mountain we&#8217;re climbing isn&#8217;t getting any less foggy.&nbsp;<\/p>\n<p>But we can choose whether to follow false maps or build better ones together.&nbsp;<\/p>\n<p>Science isn&#8217;t always about having all the answers \u2013 it&#8217;s about asking better questions.<\/p>\n<p>I like to say that changing someone else\u2019s behavior and standards takes time.&nbsp;<\/p>\n<p>In contrast, you can immediately change yours. Change begins with the person in the mirror.&nbsp;<\/p>\n<p>Whether you conduct, report, or consume SEO research.<\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" target=\"_blank\" >Technology<\/a><\/span> category.<\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/searchengineland.com\/seo-research-lie-truth-460604\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A mirage in the desert looks like water from a distance and can fool even experienced travelers into chasing something that isn\u2019t there. SEO research can be the same. It looks like science, sounds legitimate, and can trick even seasoned marketers into believing they\u2019ve found something real. Daniel Kahneman once said people would rather use&#8230;<\/p>\n","protected":false},"author":1,"featured_media":684799,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/08\/Most-SEO-research-doesnt-lie-\u2013-but-doesnt-tell-the-truth-either.jpg","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-684798","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/684798","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=684798"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/684798\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/684799"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=684798"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=684798"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=684798"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}