{"id":699747,"date":"2025-11-18T05:15:33","date_gmt":"2025-11-18T02:15:33","guid":{"rendered":"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/"},"modified":"2025-11-18T05:15:33","modified_gmt":"2025-11-18T02:15:33","slug":"small-tests-to-yield-big-answers-on-what-influences-llms-2","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/","title":{"rendered":"Small tests to yield big answers on what influences LLMs"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a33bd93361d9\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a33bd93361d9\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/#Learn_how_LLMs_pick_sources_and_how_small_content_changes_can_influence_ChatGPT_and_Google_AI_answers_See_real_tests_RAG_insights_and_practical_steps_to_optimize_your_brand_for_AI_search\" >Learn how LLMs pick sources and how small content changes can influence ChatGPT and Google AI answers. See real tests, RAG insights, and practical steps to optimize your brand for AI search.<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/#Create_branded_content_and_check_for_results\" >Create branded content and check for results<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/#Test_to_see_if_ChatGPT_is_using_Bing_or_Googles_index\" >Test to see if ChatGPT is using Bing or Google\u2019s index<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/#Change_the_content_on_a_source_used_by_RAG\" >Change the content on a source used by RAG<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/#Test_small_then_scale\" >Test small, then scale<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/#Running_LLM_tests_with_larger_websites\" >Running LLM tests with larger websites<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/buradabiliyorum.com\/en\/small-tests-to-yield-big-answers-on-what-influences-llms-2\/#Checklist_Getting_started_with_LLM_optimization\" >Checklist: Getting started with LLM optimization<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"subhead\" itemprop=\"alternativeHeadline\"><span class=\"ez-toc-section\" id=\"Learn_how_LLMs_pick_sources_and_how_small_content_changes_can_influence_ChatGPT_and_Google_AI_answers_See_real_tests_RAG_insights_and_practical_steps_to_optimize_your_brand_for_AI_search\"><\/span>Learn how LLMs pick sources and how small content changes can influence ChatGPT and Google AI answers. See real tests, RAG insights, and practical steps to optimize your brand for AI search.<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><\/p>\n<div class=\"bialty-container\">\n<p>Undoubtedly, one of the hot topics in SEO over the last few months has been how to influence LLM answers. Every SEO is trying to come up with strategies. Many have created their own tools using \u201cvibe coding,\u201d where they test their hypotheses and engage in heated debates about <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.semrush.com\/blog\/ai-overviews\/\" target=\"_blank\" rel=\"noopener\">what each LLM and Google use to pick their sources<\/a>.<\/p>\n<p>Some of these debates can get very technical, touching on topics like vector embeddings, passage ranking, retrieval-augmented generation (RAG), and chunking. These theories are great\u2014there\u2019s a lot to learn from them and turn into practice.\u00a0<\/p>\n<p>However, if some of these AI concepts are going way over your head, let\u2019s take a step back. I\u2019ll walk you through some recent tests I\u2019ve run to help you gain an understanding of what\u2019s going on in AI search without feeling overwhelmed so you can start optimizing for these new platforms.<\/p>\n<h2 id=\"create-branded-content-and-check-for-results\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Create_branded_content_and_check_for_results\"><\/span>Create branded content and check for results<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>A while ago, I went to Austin, Texas, for a business outing. Before the trip, I wondered if I could \u201cteach\u201d ChatGPT about my upcoming <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/trip-and-travel\/\" data-internallinksmanager029f6b8e52c=\"10\" title=\"Trip &amp; Travel\" target=\"_blank\" rel=\"noopener\">travel<\/a>s. There was no public information about the trip on the web, so it was a completely clean test with no competition.<\/p>\n<p>I asked ChatGPT, \u201cis Gus Pelogia going to Austin soon?\u201d The initial answer was what you\u2019d expect: He doesn\u2019t have any trips planned to Austin.<\/p>\n<p>That same day, a few hours later, I wrote a blog post on my website about my trip to Austin. Six hours after I published the post, ChatGPT\u2019s answer changed: Yes, Gus <strong>IS<\/strong> going to Austin to meet his work colleagues.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1525\" height=\"2048\" http: alt=\"Chatgpt Changed Answer Scaled\" class=\"wp-image-464228\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-changed-answer-scaled.png 1525w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-changed-answer-768x1032.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-changed-answer-1144x1536.png 1144w\" data-lazy-sizes=\"(max-width: 1525px) 100vw, 1525px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-changed-answer-scaled.png\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1525\" height=\"2048\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-changed-answer-scaled.png\" alt=\"Chatgpt Changed Answer Scaled\" class=\"wp-image-464228\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-changed-answer-scaled.png 1525w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-changed-answer-768x1032.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-changed-answer-1144x1536.png 1144w\" sizes=\"(max-width: 1525px) 100vw, 1525px\"><\/figure>\n<\/div>\n<p class=\"has-text-align-center has-small-font-size\"><em>ChatGPT prompts with a blog post published in between queries, which was enough to change a ChatGPT answer.<\/em><\/p>\n<p>ChatGPT used an AI framework called RAG (Retrieval Augmented Generation) to fetch the latest result. Basically, it didn\u2019t have enough knowledge about this information in its training data, so it scanned the web to look for an up-to-date answer.<\/p>\n<p>Interestingly enough, it took a few days until the <em>actual<\/em> blog post with detailed information was found by ChatGPT. Initially, ChatGPT had found a snippet of the new blog post on my homepage and reindexed the page within the six-hour range. It was using just the blog post\u2019s page title to change its answer before actually \u201cseeing\u201d the whole content days later.<\/p>\n<p>Some learnings from this experiment:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>New information on webpages reaches ChatGPT answers in a matter of hours, even for small websites. <\/strong>Don\u2019t think your website is too small or insignificant to get noticed by LLMs\u2014they\u2019ll notice when you add new content or refresh existing pages, so it\u2019s important to have an ongoing brand content strategy.<\/li>\n<li><strong>The answers in ChatGPT are highly dependent on the content published on your website.<\/strong> This is especially true for new companies where there are limited sources of information. ChatGPT didn\u2019t confirm that I had upcoming travel until it fetched the information from my blog post detailing the trip.<\/li>\n<li><strong>Use your webpages to optimize how your brand is portrayed beyond showing up in competitive keywords for search.<\/strong> This is your opportunity to promote a certain USP or brand tagline. For instance, \u201cThe Leading AI-Powered Marketing Platform\u201d and \u201cSee everyday moments from your close friends\u201d are used, respectively, by Semrush and Instagram on their homepages. While users probably aren\u2019t searching for these keywords, it\u2019s still an opportunity for brand positioning that will resonate with them.<\/li>\n<\/ul>\n<div style=\"background: radial-gradient(circle at 30% 40%, rgba(184, 111, 255, 0.15), rgba(0, 169, 255, 0.15) 40%, #CDE8FD 70%); padding: 30px; width: 100%; max-width: 802px; color: #000000 !important; font-family: Arial, sans-serif; margin: 25px 0 30px 0; border-radius: 8px; box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1); position: relative; box-sizing: border-box;\">\n    <!-- Headline --><\/p>\n<div style=\"width: 100%; max-width: 100%; margin-bottom: 20px; text-align: left; padding-right: 20px; box-sizing: border-box;\">\n<div class=\"headline-responsive\" style=\"font-family: Oswald, sans-serif; font-size: 32px; font-weight: normal; margin: 0; color: #000000 !important; line-height: 1.2;\">\n        <span style=\"background: linear-gradient(90deg, #D56EFE 0%, #068EF8 51%); -webkit-background-clip: text; -webkit-text-fill-color: transparent; background-clip: text;\">Win every search<\/span> with AI visibility + traditional SEO\n      <\/div>\n<p style=\"font-family: Roboto, sans-serif; font-size: 18px; font-weight: 300; line-height: 25px; margin: 12px 0 0 0; color: #000000 !important;\">\n        Built for how people search today. Track your brand across Google rankings and AI search in one place.\n      <\/p>\n<\/p><\/div>\n<p>    <!-- CTA Section --><\/p>\n<div style=\"margin-bottom: 15px;\">\n      <span style=\"display: inline-block; background-color: #FF642D; color: white; height: 44px; border: none; border-radius: 5px; cursor: pointer; font-size: 16px; padding: 0 24px; font-weight: bold; white-space: nowrap; box-sizing: border-box; text-decoration: none; line-height: 44px;\">Try free for 14 days<\/span>\n    <\/div>\n<p>    <!-- Get started with section - stacked vertically --><\/p>\n<div style=\"font-size: 12px;\">\n<div style=\"font-family: Roboto, sans-serif; font-weight: 300; color: #000000; margin-bottom: 4px;\">Get started with<\/div>\n<p>      <img loading=\"lazy\" width=\"400\" height=\"52\" decoding=\"async\" http: alt=\"Semrush One Logo\" style=\"height: 16px; width: auto; display: block;\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/semrush-one.webp\"><img loading=\"lazy\" width=\"400\" height=\"52\" decoding=\"async\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/semrush-one.webp\" alt=\"Semrush One Logo\" style=\"height: 16px; width: auto; display: block;\">\n    <\/div>\n<\/p><\/div>\n<style>\n  @media (max-width: 768px) {\n    .headline-responsive {\n      font-size: 30px !important;\n      line-height: 1.3 !important;\n    }\n  }\n<\/style>\n<h2 id=\"test-to-see-if-chatgpt-is-using-bing-or-googles-index\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Test_to_see_if_ChatGPT_is_using_Bing_or_Googles_index\"><\/span>Test to see if ChatGPT is using Bing or Google\u2019s index<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The industry has been ringing alarm bells about whether ChatGPT uses Google\u2019s index instead of Bing. So I ran another small test to find out: I added a &lt;meta name=\u201dgooglebot\u201d content=\u201dnoindex\u201d&gt; tag on the blog post, allowing only Bingbot for nine days.<\/p>\n<p>If ChatGPT is using Bing\u2019s index, it should find my new page when I prompt about it. Again, this was on a new topic and the prompt specifically asked for an article I wrote, so there wouldn\u2019t be any doubts about what source to show.<\/p>\n<p>The page got indexed by Bing after a couple of days, while Google wasn\u2019t allowed to see it.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1970\" height=\"2048\" http: alt=\"Bing Url Inspection Scaled\" class=\"wp-image-464229\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/bing-url-inspection-scaled.png 1970w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/bing-url-inspection-768x798.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/bing-url-inspection-1477x1536.png 1477w\" data-lazy-sizes=\"(max-width: 1970px) 100vw, 1970px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/bing-url-inspection-scaled.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1970\" height=\"2048\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/bing-url-inspection-scaled.png\" alt=\"Bing Url Inspection Scaled\" class=\"wp-image-464229\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/bing-url-inspection-scaled.png 1970w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/bing-url-inspection-768x798.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/bing-url-inspection-1477x1536.png 1477w\" sizes=\"auto, (max-width: 1970px) 100vw, 1970px\"><\/figure>\n<\/div>\n<p class=\"has-text-align-center has-small-font-size\"><em>New article has been indexed by Bingbot<\/em><\/p>\n<p>I kept asking ChatGPT, with multiple prompt variations, if it could find my new article. For nine days, nothing changed\u2014it couldn\u2019t find the article. It got to a point that ChatGPT hallucinated (actually, tried its best guess) a URL.<\/p>\n<p><strong>ChatGPT made-up URL:<\/strong> https:\/\/www.guspelogia.com\/learnings-from-building-a-new-product-as-an-seo<br \/><strong>Real URL:<\/strong> https:\/\/www.guspelogia.com\/learnings-new-product-seo<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1983\" http: alt=\"Gsc Page Indexing Scaled\" class=\"wp-image-464231\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/gsc-page-indexing-scaled.png 2048w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/gsc-page-indexing-768x744.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/gsc-page-indexing-1536x1487.png 1536w\" data-lazy-sizes=\"(max-width: 2048px) 100vw, 2048px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/gsc-page-indexing-scaled.png\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1983\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/gsc-page-indexing-scaled.png\" alt=\"Gsc Page Indexing Scaled\" class=\"wp-image-464231\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/gsc-page-indexing-scaled.png 2048w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/gsc-page-indexing-768x744.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/gsc-page-indexing-1536x1487.png 1536w\" sizes=\"auto, (max-width: 2048px) 100vw, 2048px\"><\/figure>\n<\/div>\n<p class=\"has-text-align-center has-small-font-size\"><em>GSC shows that it can\u2019t index the page due to \u201cnoindex\u201d tag<\/em><\/p>\n<p>I eventually gave up and allowed Googlebot to index the page. A few hours later, ChatGPT changed its answer and found the correct URL.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1252\" height=\"2048\" http: alt=\"Chatgpt Correct Url Scaled\" class=\"wp-image-464232\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-correct-url-scaled.png 1252w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-correct-url-768x1256.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-correct-url-939x1536.png 939w\" data-lazy-sizes=\"(max-width: 1252px) 100vw, 1252px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-correct-url-scaled.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1252\" height=\"2048\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-correct-url-scaled.png\" alt=\"Chatgpt Correct Url Scaled\" class=\"wp-image-464232\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-correct-url-scaled.png 1252w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-correct-url-768x1256.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-correct-url-939x1536.png 939w\" sizes=\"auto, (max-width: 1252px) 100vw, 1252px\"><\/figure>\n<\/div>\n<p class=\"has-text-align-center has-small-font-size\"><em>On the top, ChatGPT\u2019s answer when Googlebot was blocked. On the bottom, ChatGPT\u2019s answer after Googlebot was allowed to see the page.<\/em><\/p>\n<p>Interestingly enough, the link to the article was presented on my homepage and blog pages, yet ChatGPT couldn\u2019t display it. It only found that the blog post existed based on the text on those pages, even though it didn\u2019t follow the link.<\/p>\n<p>Yet, there\u2019s no harm in setting up your website for success on Bing. They\u2019re one of the search engines that adopted IndexNow, a simple ping that informs search engines that a URL\u2019s content has changed. This implementation allows Bing to reflect updates in their search results quickly.\u00a0<\/p>\n<p>While we all suspect (with evidence) that ChatGPT isn\u2019t using Bing\u2019s index, setting up IndexNow is a low effort task that\u2019s worthwhile.<\/p>\n<h2 id=\"change-the-content-on-a-source-used-by-rag\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Change_the_content_on_a_source_used_by_RAG\"><\/span>Change the content on a source used by RAG<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Clicks are becoming less important. Instead, <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.semrush.com\/blog\/ai-mentions\/\" target=\"_blank\" rel=\"noopener\">being mentioned in sources like Google\u2019s AI Mode<\/a> is arising as a new KPI for marketing teams. SEOs are testing multiple tactics to \u201cconvince\u201d LLMs about a topic. From using LinkedIn Pulse to write about a topic, to <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.rebootonline.com\/controlled-geo-experiment\/\" target=\"_blank\" rel=\"noopener\">controlled experiments with expired domains<\/a> and <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/digitaloft.co.uk\/hacked-sites-and-expired-domains-are-being-used-as-chatgpt-sources\/\" target=\"_blank\" rel=\"noopener\">hacking sites<\/a>, in some ways, it feels like old-school SEO is back.<\/p>\n<p>We\u2019re all talking about being included in AI search results, but what h<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>ens when a company or product <em>loses<\/em> a mention on a page? Imagine a specific model of earbuds is removed from a \u201ctop budget earbuds\u201d list\u2014would the product lose its mention, or would Google find a new source to back up its AI answer?\u00a0<\/p>\n<p>While the answer could always be different for each user and each situation, I ran another small test to find out.<\/p>\n<p>In a listicle that mentioned multiple certification courses, I identified one course that was no longer relevant, so I removed mentions of it from multiple pages on the same domain. I did this to keep the content relevant, so measuring the changes in AI Mode was a side effect.<\/p>\n<p>Initially, within the first few days of the course getting removed from the cited URL, it continued to be part of the AI answer for a few pre-determined prompts. Google simply found a new URL in another domain to validate its initial view.\u00a0<\/p>\n<p>However, within a week, the course disappeared from AI Mode and ChatGPT completely. Basically, even though Google found another URL validating the course listing, because the \u201coriginal source\u201d (in this case, the listicle) was updated to remove the course, Google (and, by extension, ChatGPT) subsequently updated its results as well.\u00a0\u00a0<\/p>\n<p>This experiment suggests that changing the content on the source cited by LLMs can impact the AI results. But take this conclusion with a pinch of salt, as it was a small test with a highly targeted query. I specifically had a prompt combining \u201cdomain + courses\u201d so the answer would come from one domain.<\/p>\n<p>Nonetheless, while in the real world it\u2019s unlikely one citation URL would hold all the power, I\u2019d hypothesize that losing a mention on a few high-authority pages would have the side effect of losing the mention in an AI answer.<\/p>\n<h2 id=\"test-small-then-scale\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Test_small_then_scale\"><\/span>Test small, then scale<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Tests in small and controlled environments are important for learning and give confidence that your optimization has an effect. Like everything else I do in SEO, I start with an MVP (Minimum Viable Product), learn along the way, and once\/if evidence is found, make changes at scale.<\/p>\n<p>Do you want to change the perception of a product on ChatGPT? You won\u2019t get dozens of cited sources to talk about you straight away, so you\u2019d have to reach out to each single source and request a mention. You\u2019ll quickly learn how hard it is to convince these sources to update their content and whether AI optimization becomes a pay-to-play <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/game\/\" data-internallinksmanager029f6b8e52c=\"7\" title=\"Game\" target=\"_blank\" rel=\"noopener\">game<\/a> or if it can be done organically.<\/p>\n<p>Perhaps you\u2019re a source that\u2019s mentioned often when people search for a product, like earbuds. Run your MVPs to understand how much changing your content influences AI answers before you claim your influence at scale, as the changes you make could backfire. For example, what if you stop being a source for a topic due to removing certain claims from your pages?<\/p>\n<p>There\u2019s no set time for these tests to show results. As a <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a> rule, SEOs say results take a few months to appear. In the first test on this article, it took just a few hours to see results.\u00a0<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-running-llm-tests-with-larger-websites\"><span class=\"ez-toc-section\" id=\"Running_LLM_tests_with_larger_websites\"><\/span>Running LLM tests with larger websites<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Working in large teams or on large websites can be a challenge when doing LLM testing. My suggestion is to create specific initiatives and inform all stakeholders about changes to avoid confusion later, as they might question why these changes are happening.<\/p>\n<p>One <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.seerinteractive.com\/insights\/ai-optimization-test-footers-are-back-like-2003\" target=\"_blank\" rel=\"noopener\">simple but effective test<\/a> done by SEER Interactive was to update their footer tagline.<\/p>\n<ul class=\"wp-block-list\">\n<li>From: Remote-first, Philadelphia-founded<\/li>\n<li>To: 130+ Enterprise Clients, 97% Retention Rate\u00a0<\/li>\n<\/ul>\n<p>By changing the footer, ChatGPT 5 started mentioning its new tagline within 36 hours for a prompt like \u201ctell me about Seer Interactive.\u201d I\u2019ve checked, and while every time the answer is different, they still mention the \u201c97% retention rate.\u201d<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1295\" http: alt=\"Chatgpt Footer Tagline Scaled\" class=\"wp-image-464234\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-footer-tagline-scaled.png 2048w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-footer-tagline-768x486.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-footer-tagline-1536x971.png 1536w\" data-lazy-sizes=\"(max-width: 2048px) 100vw, 2048px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-footer-tagline-scaled.png\"><img loading=\"lazy\" decoding=\"async\" width=\"2048\" height=\"1295\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-footer-tagline-scaled.png\" alt=\"Chatgpt Footer Tagline Scaled\" class=\"wp-image-464234\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-footer-tagline-scaled.png 2048w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-footer-tagline-768x486.png 768w, https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/chatgpt-footer-tagline-1536x971.png 1536w\" sizes=\"auto, (max-width: 2048px) 100vw, 2048px\"><\/figure>\n<\/div>\n<p>Imagine if you decide to change the content on a number of pages, but someone else has an optimization plan for those same pages. Always run just one test per page, as results will become less reliable if you have multiple variables.<\/p>\n<p>Make sure to research your prompts, have a tracking methodology, and spread the learnings across the company, beyond your SEO counterparts. Everyone is interested in AI right now, all the way up to C-levels.<\/p>\n<p>Another suggestion is to use a tool like Semrush\u2019s AI SEO toolkit to see the key sentiment drivers about a brand. Start with the listed \u201cAreas for Improvement\u201d\u2014this should give you plenty of ideas for tests beyond \u201cSEO Reason,\u201d as it reflects how the brand is perceived beyond organic results.<\/p>\n<h2 id=\"checklist-getting-started-with-llm-optimization\" class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Checklist_Getting_started_with_LLM_optimization\"><\/span>Checklist: Getting started with LLM optimization<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Things are changing fast with AI, and it\u2019s certainly challenging to keep up to date. There\u2019s an overload of content right now, a multitude of claims, and, I dare to say, not even the LLM platforms running them have things fully figured out.<\/p>\n<p>My recommendation is to find the sources you trust (industry <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>, events, professionals) and run your own tests using the knowledge you have. The results you find for your brands and clients are always more valuable than what others are saying.<\/p>\n<p>It\u2019s a new world of SEO and everyone is trying to figure out what works for them. The best way to follow the curve (or stay ahead of it) is to keep optimizing and documenting your changes.<\/p>\n<p>To wrap it up, here\u2019s a checklist for your LLM optimization:<\/p>\n<ul class=\"wp-block-list\">\n<li>Before starting a test, make sure your selected prompts consistently return the answer you expect (such as not mentioning your brand or a feature of your product). Otherwise, the new brand mention or link could be a coincidence, not a result of your work.<\/li>\n<li>If the same claim is made on multiple pages on your website, update them across the board to increase chances of success<\/li>\n<li>Use your own website and external sources (e.g., via digital PR) to influence your brand perception. It\u2019s unclear if users will cross-check AI answers or just trust what they\u2019re told.<\/li>\n<\/ul>\n<\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" target=\"_blank\" >Technology<\/a><\/span> category.<\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/searchengineland.com\/small-tests-to-yield-big-answers-464222\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Learn how LLMs pick sources and how small content changes can influence ChatGPT and Google AI answers. See real tests, RAG insights, and practical steps to optimize your brand for AI search. Undoubtedly, one of the hot topics in SEO over the last few months has been how to influence LLM answers. Every SEO is&#8230;<\/p>\n","protected":false},"author":1,"featured_media":699748,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/searchengineland.com\/wp-content\/seloads\/2025\/11\/small-tests-big-answers-featured-image.png","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-699747","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/699747","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=699747"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/699747\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/699748"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=699747"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=699747"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=699747"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}