{"id":640503,"date":"2024-10-04T16:00:00","date_gmt":"2024-10-04T13:00:00","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/decoding-googlebot-crawl-stats-data-in-google-search-console\/"},"modified":"2024-10-04T16:00:00","modified_gmt":"2024-10-04T13:00:00","slug":"decoding-googlebot-crawl-stats-data-in-google-search-console","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/","title":{"rendered":"#Decoding Googlebot crawl stats data in Google Search Console"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a271300bbe18\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a271300bbe18\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Dive_into_the_lesser-known_Crawl_Stats_reports_plus_how_to_monitor_Googlebot_activity_and_identify_actionable_insights_for_enterprise_sites\" >Dive into the lesser-known Crawl Stats reports, plus how to monitor Googlebot activity and identify actionable insights for enterprise sites.<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Googlebot_%E2%80%98Crawl_stats\" >Googlebot \u2018Crawl stats\u2019\u00a0<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#About_the_crawl_stats_data\" >About the crawl stats data<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Google_says_this_data_is_for_advanced_users\" >Google says this data is for advanced users<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Who_will_find_crawl_stats_most_useful\" >Who will find crawl stats most useful?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#What_to_look_for_in_this_data\" >What to look for in this data<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#When_to_look_at_this_crawl_stats\" >When to look at this crawl stats<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Caveats_about_the_crawl_stats_data\" >Caveats about the crawl stats data<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Log_the_crawl_stats_data_in_a_spreadsheet\" >Log the crawl stats data in a spreadsheet<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Remember_the_reports_boxes_are_paginated\" >Remember, the reports boxes are paginated<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Which_metrics_to_monitor_and_why\" >Which metrics to monitor and why<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Total_crawl_requests\" >Total crawl requests<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Total_download_size_byte\" >Total download size (byte)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Average_response_time_ms\" >Average response time (ms)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Crawl_requests_breakdown_by_response\" >Crawl requests breakdown by response\u00a0<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Crawl_requests_breakdown_by_file_type\" >Crawl requests breakdown by file type<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Crawl_requests_breakdown_by_crawl_purpose\" >Crawl requests breakdown by crawl purpose<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Crawl_requests_breakdown_by_Googlebot_type\" >Crawl requests breakdown by Googlebot type<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-19\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Do_you_share_these_metrics_with_developers_and_product_managers\" >Do you share these metrics with developers and product managers?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-20\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Whats_next\" >What\u2019s next?\u00a0<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-21\" href=\"https:\/\/buradabiliyorum.com\/en\/decoding-googlebot-crawl-stats-data-in-google-search-console\/#Dig_deeper\" >Dig deeper<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"subhead\" itemprop=\"alternativeHeadline\"><span class=\"ez-toc-section\" id=\"Dive_into_the_lesser-known_Crawl_Stats_reports_plus_how_to_monitor_Googlebot_activity_and_identify_actionable_insights_for_enterprise_sites\"><\/span>Dive into the lesser-known Crawl Stats reports, plus how to monitor Googlebot activity and identify actionable insights for enterprise sites.<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><\/p>\n<div class=\"bialty-container\">\nTucked away in the <em>Settings<\/em> section of Google Search Console is a report few SEO professionals discuss, but I like to monitor.<\/p>\n<p>These reports are known as Crawl Stats.<\/p>\n<p>Here, you\u2019ll find an interesting set of metrics on Googlebot crawl activity. These metrics are especially useful for websites with thousands or millions of pages.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-googlebot-crawl-stats-nbsp\"><span class=\"ez-toc-section\" id=\"Googlebot_%E2%80%98Crawl_stats\"><\/span>Googlebot \u2018Crawl stats\u2019\u00a0<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Long ago, Google Search Console had easy-to-find metrics on Googlebot crawl activity. Then, it seemingly dis<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>eared. <\/p>\n<p>In reality, it was tucked away in the <em>Settings<\/em> section.<\/p>\n<p><strong>How to access the crawl stats reports:<\/strong><\/p>\n<ul class=\"wp-block-list\">\n<li>Click on <em>Settings<\/em> at the bottom of the left navigation.<\/li>\n<li>Go to the <em>Crawl stats<\/em> section.<\/li>\n<li>Click <em>Open report<\/em>.<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1563\" height=\"1600\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Googlebot-Crawl-stats-.png.webp\" alt=\"Googlebot Crawl stats\u00a0in GSC\" class=\"wp-image-447272\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Googlebot-Crawl-stats-.png.webp 1563w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Googlebot-Crawl-stats--330x338.png.webp 330w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Googlebot-Crawl-stats--586x600.png.webp 586w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Googlebot-Crawl-stats--110x113.png.webp 110w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Googlebot-Crawl-stats--768x786.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Googlebot-Crawl-stats--1500x1536.png 1500w\" sizes=\"(max-width: 1563px) 100vw, 1563px\"><figcaption class=\"wp-element-caption\"><em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">View the report in Google Search Console<\/a><\/em><\/figcaption><\/figure>\n<\/div>\n<h2 class=\"wp-block-heading\" id=\"h-about-the-crawl-stats-data\"><span class=\"ez-toc-section\" id=\"About_the_crawl_stats_data\"><\/span>About the crawl stats data<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>As Googlebot crawls your site, Google tracks and reports on various aspects of Googlebot\u2019s activity and reports on it in <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">Google crawl stats<\/a>.<\/p>\n<p>This is where you\u2019ll find high-level statistics about Google\u2019s crawling history on your website.\u00a0<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-google-says-this-data-is-for-advanced-users\"><span class=\"ez-toc-section\" id=\"Google_says_this_data_is_for_advanced_users\"><\/span>Google says this data is for advanced users<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>The Googlebot Crawl Stats data is not for the technical SEO rookies. <\/p>\n<p>Google specifically says this data is aimed at \u201cadvanced users\u201d with thousands of pages on their site, which may be why it\u2019s located in such an unusual location, unseen by many in the SEO community.<\/p>\n<p>One reason Google may perceive this as an advanced report is that so many things can influence these metrics, including network issues and cloud delivery services such as Akamai.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-who-will-find-crawl-stats-most-useful\"><span class=\"ez-toc-section\" id=\"Who_will_find_crawl_stats_most_useful\"><\/span>Who will find crawl stats most useful?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>I find the Crawl Stats reports less of an \u201cadvanced\u201d set of reports but something that\u2019s more useful to enterprise SEOs without crawler monitoring tools such as Lumar and Botify.\u00a0<\/p>\n<p>When doing SEO on an enterprise website with thousands or millions of pages, crawler optimization is a vital task, and crawler activity metrics provide important insight for defining action items.<\/p>\n<p>Smaller sites likely do not need to worry too much about crawler activity because there is probably enough crawl budget allocated to your site to crawl at an appropriate pace.\u00a0<\/p>\n<p>On the other hand, enterprise sites tend to have far more pages that need to be crawled, discovered, and\/or refreshed than Google crawls their site each day. <\/p>\n<p>For this reason, they must optimize for crawler activity, which is a tool to help guide next steps.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-what-to-look-for-in-this-data\"><span class=\"ez-toc-section\" id=\"What_to_look_for_in_this_data\"><\/span>What to look for in this data<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>After years of reviewing this data across many sites, I have one primary rule: <\/p>\n<ul class=\"wp-block-list\">\n<li>Do not spend a lot of time here unless you see fluctuations and correlations. <\/li>\n<\/ul>\n<p>Often these reports are interesting but not actionable.<\/p>\n<p>Example fluctuations that I tend to investigate:\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>HTML requests decreased (or spiked) at the same time Bytes of JavaScript downloaded increased (or spiked).<\/li>\n<li>Average response time increased (or spiked) at the same time the number of HTML requests went down (or sudden fall).<\/li>\n<li>Total download size increased (or spiked), but the number of HTML requests did not change.<\/li>\n<li>The percent of requests for discovery (to discover new URLs) increases and the percent of requests for refresh goes down; however, you did not launch new URLs on the site.<\/li>\n<\/ul>\n<h2 class=\"wp-block-heading\" id=\"h-when-to-look-at-this-crawl-stats\"><span class=\"ez-toc-section\" id=\"When_to_look_at_this_crawl_stats\"><\/span>When to look at this crawl stats<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Crawl stats are good to peruse (and log) at least once a month. <\/p>\n<p>They are especially good to monitor after major releases, such as a platform migration or major redesign. This will help you quickly understand how Google is responding to your newly launched changes.<\/p>\n<p>Remember: If you have a bot monitoring tool such as Lumar or Botify, then this data isn\u2019t as useful as you\u2019ll find in the bot monitoring data provided by these tools.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-caveats-about-the-crawl-stats-data\"><span class=\"ez-toc-section\" id=\"Caveats_about_the_crawl_stats_data\"><\/span>Caveats about the crawl stats data<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Many things can influence Google\u2019s crawl stats metrics beyond a normal dev release. <\/p>\n<p>This means the SEO team, product manager(s) and developer(s) must think outside the box when evaluating the fluctuations.<\/p>\n<p>You must consider what could have caused an increase or decrease in Googlebot crawl activity, not only in your release but also within the network and tech stack. <\/p>\n<p>Changes to something such as Akamai could potentially impact this data.\u00a0<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-log-the-crawl-stats-data-in-a-spreadsheet\"><span class=\"ez-toc-section\" id=\"Log_the_crawl_stats_data_in_a_spreadsheet\"><\/span>Log the crawl stats data in a spreadsheet<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>This is data I like to archive because Google reports such a small window of time.<\/p>\n<p>A great example of this is a challenge I\u2019m facing now with a client. What is reported in GSC right now looks like things are improving:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1414\" height=\"586\" alt=\"Seemingly: Things are Improving\" class=\"wp-image-447274\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving.png.webp 1414w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving-600x249.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving-800x332.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving-200x83.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving-768x318.png.webp 768w\" data-lazy-sizes=\"(max-width: 1414px) 100vw, 1414px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1414\" height=\"586\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving.png.webp\" alt=\"Seemingly: Things are Improving\" class=\"wp-image-447274\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving.png.webp 1414w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving-600x249.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving-800x332.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving-200x83.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/GSC-Seemingly-Things-are-Improving-768x318.png.webp 768w\" sizes=\"auto, (max-width: 1414px) 100vw, 1414px\"><figcaption class=\"wp-element-caption\"><em><em>Seemingly, things are improving<\/em><\/em><\/figcaption><\/figure>\n<\/div>\n<p>However, because I have metrics from six months ago, I can say that these metrics are 40% higher than they were six months ago.<\/p>\n<p>While they\u2019re trending down, they\u2019re still worse than they were in the past. The client\u2019s challenge is that development has no idea why this is happening (unfortunately, solving that problem is beyond the scope of this article).<\/p>\n<p>You may think to just grab a screenshot. However, it makes it very hard to compare over time. <\/p>\n<p>Notice there is no left axis in the chart. You really cannot tell what the lines reflect. (<strong>Note<\/strong>: Numbers do appear on the left\/right axis when you are only viewing two metrics in the chart)<\/p>\n<p>Instead, drop this data into a spreadsheet. Then, you have actual data that can be charted over time, calculated and used to compare with other metrics, such as visits.\u00a0<\/p>\n<p>Having the historical data in one place is often useful when discussing major changes with development to show how much better the metrics were 4-6+ months ago.\u00a0<\/p>\n<p>Remember, development likes hard, specific data, so charts with actual numbers on the left\/right axis (or worse, no numbers on the <em>x<\/em>-axis at all) will be more useful to you than charts with varying numbers on the <em>x<\/em>-axis.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-remember-the-reports-boxes-are-paginated\"><span class=\"ez-toc-section\" id=\"Remember_the_reports_boxes_are_paginated\"><\/span>Remember, the reports boxes are paginated<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Though the most important metrics you\u2019ll need are likely visible in the default view, many of the report sections are paginated \u2013 and they\u2019re easy to miss!\u00a0<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"918\" height=\"854\" alt=\"Paginated reports boxes\" class=\"wp-image-447275\" style=\"width:629px;height:auto\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes.png.webp 918w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes-363x338.png.webp 363w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes-645x600.png.webp 645w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes-121x113.png.webp 121w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes-768x714.png.webp 768w\" data-lazy-sizes=\"(max-width: 918px) 100vw, 918px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"918\" height=\"854\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes.png.webp\" alt=\"Paginated reports boxes\" class=\"wp-image-447275\" style=\"width:629px;height:auto\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes.png.webp 918w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes-363x338.png.webp 363w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes-645x600.png.webp 645w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes-121x113.png.webp 121w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Paginated-reports-boxes-768x714.png.webp 768w\" sizes=\"auto, (max-width: 918px) 100vw, 918px\"><\/figure>\n<\/div>\n<p><!-- START INLINE FORM --><\/p>\n<p><!-- END INLINE FORM --><\/p>\n<hr class=\"wp-block-separator has-text-color has-cyan-bluish-gray-color has-css-opacity has-cyan-bluish-gray-background-color has-background\">\n<h2 class=\"wp-block-heading\" id=\"h-which-metrics-to-monitor-and-why\"><span class=\"ez-toc-section\" id=\"Which_metrics_to_monitor_and_why\"><\/span>Which metrics to monitor and why<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Let\u2019s get into the primary metrics to look (very quickly) each month, along with a few tips to take away action items from the data:<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-total-crawl-requests\"><span class=\"ez-toc-section\" id=\"Total_crawl_requests\"><\/span>Total crawl requests<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul class=\"wp-block-list\">\n<li><em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">View this report in Google Search Console <\/a>(located in the top chart).<\/em><\/li>\n<li><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/support.google.com\/webmasters\/answer\/9679690?hl=en#total_crawl_requests\">Google definition<\/a>: \u201cThe total number of crawl requests issued for URLs on your site, whether successful or not.\u201d<\/li>\n<li>If this metric goes up or down, compare it with average response time and total download size (bytes).\n<ul class=\"wp-block-list\">\n<li>An obvious reason for this metric could go up if you change a lot of code or launch a lot of new pages. However, that is by no means the only cause.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1418\" height=\"610\" alt=\"Total crawl requests\" class=\"wp-image-447276\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests.png.webp 1418w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests-600x258.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests-800x344.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests-200x86.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests-768x330.png.webp 768w\" data-lazy-sizes=\"(max-width: 1418px) 100vw, 1418px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1418\" height=\"610\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests.png.webp\" alt=\"Total crawl requests\" class=\"wp-image-447276\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests.png.webp 1418w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests-600x258.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests-800x344.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests-200x86.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-crawl-requests-768x330.png.webp 768w\" sizes=\"auto, (max-width: 1418px) 100vw, 1418px\"><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\" id=\"h-total-download-size-byte\"><span class=\"ez-toc-section\" id=\"Total_download_size_byte\"><\/span>Total download size (byte)<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul class=\"wp-block-list\">\n<li><em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">View this report in Google Search Console<\/a> (located in the top chart).<\/em><\/li>\n<li><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/support.google.com\/webmasters\/answer\/9679690?hl=en#total_download_size\">Google definition<\/a>: \u201cTotal number of bytes downloaded from your site during crawling, for the specified time period.\u201d<\/li>\n<li>If this metric goes up or down, compare it with average response time<\/li>\n<li>An obvious cause for this metric to increase is adding a lot of code across thousands of pages or launching a lot of new pages. However, that is by no means the only cause.<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1400\" height=\"604\" alt=\"Total download size (byte)\" class=\"wp-image-447277\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte.png.webp 1400w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte-600x259.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte-800x345.png 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte-200x86.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte-768x331.png 768w\" data-lazy-sizes=\"(max-width: 1400px) 100vw, 1400px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1400\" height=\"604\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte.png.webp\" alt=\"Total download size (byte)\" class=\"wp-image-447277\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte.png.webp 1400w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte-600x259.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte-800x345.png 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte-200x86.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Total-download-size-byte-768x331.png 768w\" sizes=\"auto, (max-width: 1400px) 100vw, 1400px\"><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\" id=\"h-average-response-time-ms\"><span class=\"ez-toc-section\" id=\"Average_response_time_ms\"><\/span>Average response time (ms)<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul class=\"wp-block-list\">\n<li><em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">Google Search Console Report<\/a> (located in the top chart).<\/em><\/li>\n<li><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/support.google.com\/webmasters\/answer\/9679690?hl=en#avg_response_time\">Google definition<\/a>: \u201cAverage response time for all resources fetched from your site during the specified time period.\u201d<\/li>\n<li>If this metric goes up or down, compare with with total crawl requests and total download size (bytes).<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1418\" height=\"610\" alt=\"Average response time (ms)\" class=\"wp-image-447278\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms.png.webp 1418w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms-600x258.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms-800x344.png 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms-200x86.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms-768x330.png.webp 768w\" data-lazy-sizes=\"(max-width: 1418px) 100vw, 1418px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1418\" height=\"610\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms.png.webp\" alt=\"Average response time (ms)\" class=\"wp-image-447278\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms.png.webp 1418w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms-600x258.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms-800x344.png 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms-200x86.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Average-response-time-ms-768x330.png.webp 768w\" sizes=\"auto, (max-width: 1418px) 100vw, 1418px\"><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\" id=\"h-crawl-requests-breakdown-by-response-nbsp\"><span class=\"ez-toc-section\" id=\"Crawl_requests_breakdown_by_response\"><\/span>Crawl requests breakdown by response\u00a0<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul class=\"wp-block-list\">\n<li><em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">View this report in Google Search Console<\/a> (located below the top chart).<\/em><\/li>\n<li><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/support.google.com\/webmasters\/answer\/9679690?hl=en#response\">Google definition<\/a>: \u201cThis table shows the responses that Google received when crawling your site, grouped by response type, as a percentage of all crawl responses\u2026\u201d<\/li>\n<li>Common responses:\n<ul class=\"wp-block-list\">\n<li>OK (200).<\/li>\n<li>Moved permanently (302).<\/li>\n<li>Server error (5xx).<\/li>\n<li>Other client error (4xx).<\/li>\n<li>Not found (404).<\/li>\n<li>Not modified (304).<\/li>\n<li>Page timeout.<\/li>\n<li>Robots.txt not available.<\/li>\n<li>Redirect error.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"704\" height=\"590\" alt=\"Crawl requests breakdown by response\u00a0\" class=\"wp-image-447279\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-response.png.webp 704w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-response-403x338.png.webp 403w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-response-135x113.png.webp 135w\" data-lazy-sizes=\"(max-width: 704px) 100vw, 704px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-response.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"704\" height=\"590\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-response.png.webp\" alt=\"Crawl requests breakdown by response\u00a0\" class=\"wp-image-447279\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-response.png.webp 704w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-response-403x338.png.webp 403w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-response-135x113.png.webp 135w\" sizes=\"auto, (max-width: 704px) 100vw, 704px\"><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\" id=\"h-crawl-requests-breakdown-by-file-type\"><span class=\"ez-toc-section\" id=\"Crawl_requests_breakdown_by_file_type\"><\/span>Crawl requests breakdown by file type<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul class=\"wp-block-list\">\n<li><em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">View this report in Google Search Console<\/a>.\u00a0<\/em><\/li>\n<li><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/support.google.com\/webmasters\/answer\/9679690?hl=en#file_type\">Google definition<\/a>: \u201cThe file type returned by the request. Percentage value for each type is the percentage of responses of that type, not the percentage of of bytes retrieved of that type.\u201d<\/li>\n<li>Common responses:\n<ul class=\"wp-block-list\">\n<li>JSON.<\/li>\n<li>HTML.<\/li>\n<li>JavaScript.<\/li>\n<li>Image.<\/li>\n<li>PDF.<\/li>\n<li>CSS.<\/li>\n<li>Syndication.<\/li>\n<li>Other XML.<\/li>\n<li>Video.<\/li>\n<li>Other file type.<\/li>\n<li>Unknown (failed requests).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"702\" height=\"588\" alt=\"Crawl requests breakdown by file type\" class=\"wp-image-447280\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-file-type.png.webp 702w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-file-type-404x338.png 404w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-file-type-135x113.png.webp 135w\" data-lazy-sizes=\"(max-width: 702px) 100vw, 702px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-file-type.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"702\" height=\"588\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-file-type.png.webp\" alt=\"Crawl requests breakdown by file type\" class=\"wp-image-447280\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-file-type.png.webp 702w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-file-type-404x338.png 404w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-file-type-135x113.png.webp 135w\" sizes=\"auto, (max-width: 702px) 100vw, 702px\"><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\" id=\"h-crawl-requests-breakdown-by-crawl-purpose\"><span class=\"ez-toc-section\" id=\"Crawl_requests_breakdown_by_crawl_purpose\"><\/span>Crawl requests breakdown by crawl purpose<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul class=\"wp-block-list\">\n<li><em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">View this report in Google Search Console<\/a>.<\/em><\/li>\n<li><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/support.google.com\/webmasters\/answer\/9679690?hl=en#purpose\">Two purposes<\/a>:\n<ul class=\"wp-block-list\">\n<li>Refresh.<\/li>\n<li>Discovery.<\/li>\n<\/ul>\n<\/li>\n<li>This is an interesting metric for presentations; however, it only has a few useful use cases. For example:\n<ul class=\"wp-block-list\">\n<li>If the percent of Googlebot activity that is for <em>Discovery<\/em> suddenly increases, but we\u2019re not adding URLs to the site, then you have an action item to figure out what is being crawled that shouldn\u2019t be crawled.\u00a0<\/li>\n<li>If the percent of Googlebot activity that is for <em>Refresh<\/em> decreases significantly, but you didn\u2019t remove pages from the site, then you have an action item to figure out why fewer existing pages are being crawled.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"706\" height=\"364\" alt=\"Crawl requests breakdown by crawl purpose\" class=\"wp-image-447282\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-crawl-purpose.png.webp 706w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-crawl-purpose-600x309.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-crawl-purpose-200x103.png.webp 200w\" data-lazy-sizes=\"(max-width: 706px) 100vw, 706px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-crawl-purpose.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"706\" height=\"364\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-crawl-purpose.png.webp\" alt=\"Crawl requests breakdown by crawl purpose\" class=\"wp-image-447282\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-crawl-purpose.png.webp 706w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-crawl-purpose-600x309.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-crawl-purpose-200x103.png.webp 200w\" sizes=\"auto, (max-width: 706px) 100vw, 706px\"><\/figure>\n<\/div>\n<h3 class=\"wp-block-heading\" id=\"h-crawl-requests-breakdown-by-googlebot-type\"><span class=\"ez-toc-section\" id=\"Crawl_requests_breakdown_by_Googlebot_type\"><\/span>Crawl requests breakdown by Googlebot type<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul class=\"wp-block-list\">\n<li><em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/search.google.com\/search-console\/settings\/crawl-stats\">View this report in Google Search Console<\/a>.<\/em><\/li>\n<li><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/support.google.com\/webmasters\/answer\/9679690?hl=en#googlebot_type\">Google definition<\/a>: \u201cThe type of user agent used to make the crawl request. Google has a number of user agents that crawl for different reasons and have different behaviors.\u201d<\/li>\n<li>It\u2019s an interesting metric, but not very useful. It just shows Google is still using their desktop crawler. Honestly, I usually ignore these metrics.<\/li>\n<\/ul>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"696\" height=\"574\" alt=\"Crawl requests breakdown by Googlebot type\" class=\"wp-image-447283\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-Googlebot-type.png.webp 696w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-Googlebot-type-410x338.png.webp 410w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-Googlebot-type-137x113.png.webp 137w\" data-lazy-sizes=\"(max-width: 696px) 100vw, 696px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-Googlebot-type.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"696\" height=\"574\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-Googlebot-type.png.webp\" alt=\"Crawl requests breakdown by Googlebot type\" class=\"wp-image-447283\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-Googlebot-type.png.webp 696w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-Googlebot-type-410x338.png.webp 410w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-breakdown-by-Googlebot-type-137x113.png.webp 137w\" sizes=\"auto, (max-width: 696px) 100vw, 696px\"><\/figure>\n<\/div>\n<p><strong>You can click into each metric for more data<\/strong><\/p>\n<p>Often when you present any SEO concern to product managers and developers, they often want to see example URLs. You can click on any of the metrics listed in this report and get example URLs.<\/p>\n<p>An interesting metric to look at is \u201cother file types\u201d because it\u2019s not clear what\u2019s in the \u201cother file types\u201d category (often it\u2019s font files).<\/p>\n<p>The screenshot below shows the examples report for \u201cother file type.\u201d Every file listed is a font file (blurred out for confidentiality reasons).<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1178\" height=\"1540\" alt=\"Crawl requests - other file types\" class=\"wp-image-447284\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types.png.webp 1178w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-259x338.png.webp 259w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-459x600.png.webp 459w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-86x113.png.webp 86w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-768x1004.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-1175x1536.png 1175w\" data-lazy-sizes=\"(max-width: 1178px) 100vw, 1178px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1178\" height=\"1540\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types.png.webp\" alt=\"Crawl requests - other file types\" class=\"wp-image-447284\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types.png.webp 1178w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-259x338.png.webp 259w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-459x600.png.webp 459w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-86x113.png.webp 86w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-768x1004.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Crawl-requests-other-file-types-1175x1536.png 1175w\" sizes=\"auto, (max-width: 1178px) 100vw, 1178px\"><\/figure>\n<\/div>\n<p>In this report of examples, each row reflects one crawl request. This means if a page is crawled multiple times it could be listed more than once in the \u201cexamples.\u201d\u00a0<\/p>\n<p>As with all Google Search Console reports, this is a data sample and not every request from Googlebot.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-do-you-share-these-metrics-with-developers-and-product-managers\"><span class=\"ez-toc-section\" id=\"Do_you_share_these_metrics_with_developers_and_product_managers\"><\/span>Do you share these metrics with developers and product managers?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>These metrics will typically generate one of two thoughts:\u00a0<\/p>\n<ul class=\"wp-block-list\">\n<li>\u201cThere\u2019s nothing to look at here.\u201d\u00a0<\/li>\n<li>\u201cWhat could have caused that?\u201d\u00a0<\/li>\n<\/ul>\n<p>In my experience, the answers to \u201cwhat caused that\u201d tend to require the input of product managers and\/or developers.<\/p>\n<p>When presenting the data and your questions about potential causes for issues, remember to clearly explain that these metrics are not user activity and solely represent Googlebot\u2019s activity and experience on the website.<\/p>\n<p>I find product managers and developers often get a bit confused when discussing this data, especially if it doesn\u2019t match up with other metrics they have seen or facts they know about the site.\u00a0<\/p>\n<p>By the way, this often happens for most Google Search Console data conversations.<\/p>\n<p>If there are no Crawl Stats fluctuations or correlations to be concerned about, don\u2019t bring it up to development, nor product management. It just becomes noise and prevents them from focusing on more critical metrics.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-what-s-next-nbsp\"><span class=\"ez-toc-section\" id=\"Whats_next\"><\/span>What\u2019s next?\u00a0<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Check out your crawl stats to make sure there are no spikes or correlations that are concerning.\u00a0<\/p>\n<p>Then, determine how often you want to look at these and set up systems that prompt you to check these and other Google Search Console metrics in a systematic, analytical method each month.<\/p>\n<p>While you check out your Googlebot Crawl Stats, I\u2019ll write Part 4 in this <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/watch-movies-tv-seriess\/\" data-internallinksmanager029f6b8e52c=\"8\" title=\"Watch Movies &amp; TV Series\" target=\"_blank\" rel=\"noopener\">series<\/a> that will talk about how to know which URLs you should focus on for technical SEO improvements and in particular, Core Web Vitals metrics.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-dig-deeper\"><span class=\"ez-toc-section\" id=\"Dig_deeper\"><\/span>Dig deeper<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>This is the third article in a series recapping my SMX Advanced presentation on how to turn SEO metrics into action items. Below are links to the first two articles:<\/p>\n<ul class=\"wp-block-list\">\n<li>Evaluating technical SEO data: Key segmentation approaches<\/li>\n<li>Key Google Search Console metrics to monitor every month<\/li>\n<\/ul>\n<\/div>\n<p><\/p>\n<div class=\"about-author\">\n    About the author<\/p>\n<div class=\"information\">\n<div class=\"author-module\">\n<div class=\"row\">\n<div class=\"col-12 col-lg-3 text-center\">\n<div class=\"avatar\">\n                        <img loading=\"lazy\" decoding=\"async\" class=\"img-fluid rounded-circle avatar-border\" alt=\"Jessica Bowman\" width=\"140\" height=\"140\" src=\"https:\/\/searchengineland.com\/images\/authors\/JessicaBowman-lg.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"img-fluid rounded-circle avatar-border\" src=\"https:\/\/searchengineland.com\/images\/authors\/JessicaBowman-lg.jpg\" alt=\"Jessica Bowman\" width=\"140\" height=\"140\">\n                                            <\/div>\n<\/p><\/div>\n<div class=\"col-12 col-lg-9\">\n<div class=\"about\">\n<div class=\"name\">\n                            <strong>Jessica Bowman<\/strong>\n                        <\/div>\n<div class=\"row g-2 pt-2\">\n<div class=\"col-auto twitter\">\n                                    <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/intent\/follow?original_referer=https%3A%2F%2Fsearchengineland.com%2F&amp;region=follow_link&amp;screen_name=jessicabowman&amp;tw_p=followbutton&amp;variant=2.0\" aria-label=\"opens in a new tab\"><i class=\"fab fa-x-twitter\"><\/i><\/a>\n                            <\/div>\n<div class=\"col-auto\">\n                                    <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/in\/jessicabowman\/\" aria-label=\"opens in a new tab\"><i class=\"fab fa-linkedin\"><\/i><\/a>\n                                <\/div>\n<\/p><\/div>\n<p>                        Enterprise SEO in-house has been putting food on Jessica Bowman&#8217;s table since 2002. She&#8217;s the trusted advisor for some of the biggest brands in the world and quickly mobilizes teams company-wide to start taking action on SEO within days from developers to product managers to merchandisers. Jessica created the only online, enterprise-level company-wide SEO training uniquely designed for large corporations to mobilize an SEO army across all roles, company-wide. Jessica founded SEOinhouse.com, a consultancy focused on doing SEO in-house and she literally wrote the playbook on doing SEO at a large company.                   <\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology<\/a><\/span> category.<\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/searchengineland.com\/google-search-console-googlebot-crawl-stats-447267\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Dive into the lesser-known Crawl Stats reports, plus how to monitor Googlebot activity and identify actionable insights for enterprise sites. Tucked away in the Settings section of Google Search Console is a report few SEO professionals discuss, but I like to monitor. These reports are known as Crawl Stats. Here, you\u2019ll find an interesting set&#8230;<\/p>\n","protected":false},"author":1,"featured_media":640504,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/10\/Googlebot-Crawl-stats-.png","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[141362,78070,148084],"class_list":["post-640503","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology","tag-google-search-console","tag-seo","tag-technical-optimization"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/640503","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=640503"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/640503\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/640504"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=640503"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=640503"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=640503"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}