{"id":644668,"date":"2024-11-19T18:00:00","date_gmt":"2024-11-19T15:00:00","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/"},"modified":"2024-11-19T18:00:00","modified_gmt":"2024-11-19T15:00:00","slug":"how-to-create-article-outlines-with-ai-based-on-googles-top-10","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/","title":{"rendered":"#How to create article outlines with AI based on Google\u2019s Top 10"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_89 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6aeeee5a86732\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6aeeee5a86732\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Use_this_guide_and_script_to_automate_SEO_content_research_to_generate_article_outlines_using_AI_APIs_and_top-ranking_page_analysis\" >Use this guide and script to automate SEO content research to generate article outlines using AI, APIs and top-ranking page analysis.<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Why_use_summaries_entities_and_top_questions_for_article_outlines\" >Why use summaries, entities and top questions for article outlines?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Summaries\" >Summaries<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Entities\" >Entities<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Top_questions\" >Top questions<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Getting_started\" >Getting started<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Step_1_Getting_your_API_keys\" >Step 1: Getting your API keys<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#What_is_an_API\" >What is an API?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Custom_search_and_Google_Cloud_Natural_Language_APIs\" >Custom search and Google Cloud Natural Language APIs<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Google_API_key\" >Google API key<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Google_service_account\" >Google service account<\/a><\/li><\/ul><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Step_2_Defining_the_article_you_want_an_outline_for\" >Step 2: Defining the article you want an outline for<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Step_3_Installing_the_libraries_youll_need\" >Step 3: Installing the libraries you\u2019ll need<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Step_4_Adding_our_APIs\" >Step 4: Adding our APIs<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Step_5_Defining_the_functions\" >Step 5: Defining the functions<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Step_6_Scraping_the_top_10_and_collecting_the_information\" >Step 6: Scraping the top 10 and collecting the information<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Step_7_Generating_our_article_outline\" >Step 7: Generating our article outline<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-article-outlines-with-ai-based-on-googles-top-10\/#Just_the_beginning\" >Just the beginning<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"subhead\" itemprop=\"alternativeHeadline\"><span class=\"ez-toc-section\" id=\"Use_this_guide_and_script_to_automate_SEO_content_research_to_generate_article_outlines_using_AI_APIs_and_top-ranking_page_analysis\"><\/span>Use this guide and script to automate SEO content research to generate article outlines using AI, APIs and top-ranking page analysis. <span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><\/p>\n<div class=\"bialty-container\">\n<p>The world of SEO has gotten increasingly nuanced. If you\u2019re here, you know that <strong>merely<\/strong> writing an article and <s>buying<\/s> links to it hasn\u2019t worked in a long while.\u00a0<\/p>\n<p>When I\u2019m working on my own articles or with authors, I start with outlines. <\/p>\n<p>After all, it\u2019s easier to optimize pieces built to rank than to reverse engineer what you want later.\u00a0<\/p>\n<p>I\u2019ve often focused a lot of my attention on understanding the content and structures of the top-ranking pages. Specifically, what questions they answered and what entities they contained. <\/p>\n<p>I did a lot of this manually. And it took a lot of time.\u00a0<\/p>\n<p>Thankfully, with the rise of LLMs, a couple of APIs and a new scraper, we\u2019re able to automate a lot of this. This can reduce the amount of time spent creating content structures that will rank. You can spend that saved time adding more of the insights that only you, as a human, can provide.<\/p>\n<p>This article will walk you through a <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">script<\/a> to create article outlines based on:<\/p>\n<ul class=\"wp-block-list\">\n<li>The keywords you\u2019re targeting.<\/li>\n<li>The type of article you want to write.<\/li>\n<li>The top entities that appear on the top-ranking websites.<\/li>\n<li>The top questions answered by the top-ranking sites.<\/li>\n<li>Summaries of the top-ranking sites.<\/li>\n<\/ul>\n<p>And if you\u2019re not interested in coding yourself, I\u2019ve even written a Google Colab you can use. You only need to sign up for the APIs (easy) and click some play buttons.<\/p>\n<p>You\u2019ll find the Colab <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/colab.research.google.com\/drive\/1PRZzS7WWJNS92bvO4vl8wOh1ZCsBrrKG\" target=\"_blank\" rel=\"noopener\">here<\/a>.<\/p>\n<p>Before we dive into the code, here\u2019s why I base my article outlines on these features.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-why-use-summaries-entities-and-top-questions-for-article-outlines\"><span class=\"ez-toc-section\" id=\"Why_use_summaries_entities_and_top_questions_for_article_outlines\"><\/span>Why use summaries, entities and top questions for article outlines?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>We could use a number of features to inspire our outlines. I chose these three because they form the foundation of what should be included in content inspired by the top 10. Here\u2019s why:<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-summaries\"><span class=\"ez-toc-section\" id=\"Summaries\"><\/span>Summaries<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Summaries help you distill what the top-ranking pages are doing well, including how they address search intent. <\/p>\n<p>Keen eyes will notice that I\u2019ve focused the summaries around the heading tags in the scripts. This ensures that when we teach the system to create an outline, it will look at the heading structures of the top-ranking sites.<\/p>\n<p>We\u2019ll be using a newer library called Firecrawl, which puts the content in a markdown format. This allows us to base our understanding of the content on more advanced structural elements, including headings. This is a big leap forward.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-entities\"><span class=\"ez-toc-section\" id=\"Entities\"><\/span>Entities<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Entities are specific, well-defined concepts, such as \u201cartificial intelligence,\u201d \u201cNew York City\u201d or even an idea like \u201ctrust.\u201d Google recognizes entities to better understand content at a deeper level, beyond just keywords.<\/p>\n<p>For SEO, this means Google connects entities within your content to related ideas, creating a web of associations. Including the top entities relevant to your topic helps you align your article with what Google sees as contextually important and related.<\/p>\n<p>I rank the entities by their salience scores, which indicate how important they are to the page\u2019s content.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-top-questions\"><span class=\"ez-toc-section\" id=\"Top_questions\"><\/span>Top questions<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Top questions are another key piece because they represent what real users want to know.<\/p>\n<p>My thought is that, by extension, questions Google wants answered. By answering these questions within the article, you will likely satisfy search intent more fully.\u00a0<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-getting-started\"><span class=\"ez-toc-section\" id=\"Getting_started\"><\/span>Getting started<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>You can run this script locally on your machine using an IDE (Integrated Development Environment). <\/p>\n<p>The remainder of this article is written for that environment. <\/p>\n<p>The great <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>? You can have this all set up and running on your machine in just a few minutes.<\/p>\n<p>If you\u2019ve never used an IDE before, you can start by installing Anaconda. It does many things, but we only want to use Jupyter Notebook for now.<\/p>\n<p>Download and install Anaconda <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.anaconda.com\/download\">here<\/a>, after which you can simply launch \u201cjupyter\u201d to get going.<\/p>\n<p>Once you\u2019ve done that, you can jump into the tutorial. <\/p>\n<p>This route will give you an easier overall experience. You can save your API keys locally and not have to enter them each time. Plus, you can edit the prompts and other variables more easily.<\/p>\n<p>Alternatively, you can simply <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/colab.research.google.com\/drive\/1PRZzS7WWJNS92bvO4vl8wOh1ZCsBrrKG\" target=\"_blank\" rel=\"noopener\">run the Colab<\/a>. This is also an easy way to test whether it\u2019s worth setting up locally.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-step-1-getting-your-api-keys\"><span class=\"ez-toc-section\" id=\"Step_1_Getting_your_API_keys\"><\/span>Step 1: Getting your API keys<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>First, you\u2019ll need to sign up for the API keys you\u2019ll need.\u00a0<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-what-is-an-api\"><span class=\"ez-toc-section\" id=\"What_is_an_API\"><\/span>What is an API?<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>APIs (application programming interfaces) are essentially pipelines that let different software systems communicate and share data.<\/p>\n<p>Think of them as a structured way to request specific data or functions from a service (like Google or OpenAI) and get back exactly what you need, whether that\u2019s analyzing entities on a page, generating text or scraping content.<\/p>\n<p>The APIs we will need are:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Custom search and Cloud Natural Language APIs (see below):<\/strong> This is to manage your custom search engine and extract entities from pages. It is paid, but my bill using it a lot is just a couple dollars per month.<\/li>\n<li><strong>OpenAI:<\/strong> This is for content analysis and creation via GPT-4o. The API is low-cost (a dollar or two per month under regular use) and has a free trial.<\/li>\n<li><strong>Firecrawl:<\/strong> This is for scraping webpages. As this level, the free option should work just fine, but if you start using it a ton they do have inexpensive options available.<\/li>\n<li><strong>Weights &amp; Biases <\/strong><em>(disclosure: I\u2019m the head of SEO at Weights &amp; Biases)<\/em><strong>:<\/strong> Sign up and obtain your API key. The free option will do everything we need.<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\" id=\"h-custom-search-and-google-cloud-natural-language-apis\"><span class=\"ez-toc-section\" id=\"Custom_search_and_Google_Cloud_Natural_Language_APIs\"><\/span>Custom search and Google Cloud Natural Language APIs<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>I found getting these APIs set up a bit non-intuitive, so I wanted to save you the 10 minutes of figuring it out. Here\u2019s a step-by-step guide to getting yourself setup with the Google APIs:<\/p>\n<p>To set up your search engine, just follow the quick instructions at <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/developers.google.com\/custom-search\/docs\/tutorial\/creatingcse\" target=\"_blank\" rel=\"noopener\"> <\/a><a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/developers.google.com\/custom-search\/docs\/tutorial\/creatingcse\">https:\/\/developers.google.com\/custom-search\/docs\/tutorial\/creatingcse.<\/a><\/p>\n<p>The settings I use are:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img fetchpriority=\"high\" decoding=\"async\" width=\"596\" height=\"687\" alt=\"Custom search and Google Cloud Natural Language APIs\" class=\"wp-image-448390\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs.png.webp 596w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs-293x338.png.webp 293w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs-521x600.png.webp 521w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs-98x113.png.webp 98w\" data-lazy-sizes=\"(max-width: 596px) 100vw, 596px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs.png.webp\"><img fetchpriority=\"high\" decoding=\"async\" width=\"596\" height=\"687\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs.png.webp\" alt=\"Custom search and Google Cloud Natural Language APIs\" class=\"wp-image-448390\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs.png.webp 596w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs-293x338.png.webp 293w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs-521x600.png.webp 521w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Custom-search-and-Google-Cloud-Natural-Language-APIs-98x113.png.webp 98w\" sizes=\"(max-width: 596px) 100vw, 596px\"><\/figure>\n<\/div>\n<h4 class=\"wp-block-heading\" id=\"h-google-api-key\"><span class=\"ez-toc-section\" id=\"Google_API_key\"><\/span><strong>Google API key<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>The API key gives you access to the search engine. Again, it\u2019s easy to set up, and you can do so at<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/support.google.com\/googleapi\/answer\/6158862?hl=en\" target=\"_blank\" rel=\"noopener\"> https:\/\/support.google.com\/googleapi\/answer\/6158862?hl=en.<\/a><\/p>\n<p>When you\u2019re in the console, you\u2019ll simply click to <em>Enable APIs<\/em> <em>and services<\/em>:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"925\" height=\"262\" alt=\"Google Cloud - Enable APIs and services\" class=\"wp-image-448391\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services.png.webp 925w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services-600x170.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services-800x227.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services-200x57.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services-768x218.png.webp 768w\" data-lazy-sizes=\"(max-width: 925px) 100vw, 925px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"925\" height=\"262\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services.png.webp\" alt=\"Google Cloud - Enable APIs and services\" class=\"wp-image-448391\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services.png.webp 925w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services-600x170.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services-800x227.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services-200x57.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Enable-APIs-and-services-768x218.png.webp 768w\" sizes=\"auto, (max-width: 925px) 100vw, 925px\"><\/figure>\n<\/div>\n<p>And you\u2019ll want to enable the <em>Custom Search API<\/em> and <em>Cloud Natural Language API<\/em>.<\/p>\n<p>You\u2019ll also need to set up your credentials:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"963\" height=\"418\" alt=\"Google Cloud - Set up your credentials\" class=\"wp-image-448393\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials.png.webp 963w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials-600x260.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials-800x347.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials-200x87.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials-768x333.png.webp 768w\" data-lazy-sizes=\"(max-width: 963px) 100vw, 963px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"963\" height=\"418\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials.png.webp\" alt=\"Google Cloud - Set up your credentials\" class=\"wp-image-448393\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials.png.webp 963w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials-600x260.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials-800x347.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials-200x87.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Set-up-your-credentials-768x333.png.webp 768w\" sizes=\"auto, (max-width: 963px) 100vw, 963px\"><\/figure>\n<\/div>\n<p>Then, select <em>API key<\/em> from the drop-down:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"694\" height=\"342\" alt=\"Google Cloud - API key credentials\" class=\"wp-image-448394\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-API-key-credentials.png.webp 694w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-API-key-credentials-600x296.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-API-key-credentials-200x99.png.webp 200w\" data-lazy-sizes=\"(max-width: 694px) 100vw, 694px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-API-key-credentials.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"694\" height=\"342\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-API-key-credentials.png.webp\" alt=\"Google Cloud - API key credentials\" class=\"wp-image-448394\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-API-key-credentials.png.webp 694w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-API-key-credentials-600x296.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-API-key-credentials-200x99.png.webp 200w\" sizes=\"auto, (max-width: 694px) 100vw, 694px\"><\/figure>\n<\/div>\n<p>Copy the key to a Notepad file. You can save it there, but we\u2019ll need it in a moment, so I <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a>ly paste mine in an unsaved doc.<\/p>\n<p>For good measure, I recommend clicking on the API Key you just created. It will have an orange triangle beside it, noting that it\u2019s unrestricted. <\/p>\n<p>You can click it, set the API to restricted and give it access to just the Custom Search API to help safeguard against misuse.<\/p>\n<h4 class=\"wp-block-heading\" id=\"h-google-service-account\"><span class=\"ez-toc-section\" id=\"Google_service_account\"><\/span><strong>Google service account<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>While you\u2019re on this screen, you can set up the service account.<\/p>\n<p>Again, you\u2019ll click on<em> Create credentials<\/em>, but instead of <em>API key<\/em>, you\u2019ll click <em>Service account<\/em>.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"852\" height=\"362\" alt=\"Google Cloud - Service acocunt credentials\" class=\"wp-image-448395\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials.png.webp 852w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials-600x255.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials-800x340.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials-200x85.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials-768x326.png.webp 768w\" data-lazy-sizes=\"(max-width: 852px) 100vw, 852px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"852\" height=\"362\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials.png.webp\" alt=\"Google Cloud - Service acocunt credentials\" class=\"wp-image-448395\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials.png.webp 852w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials-600x255.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials-800x340.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials-200x85.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Service-acocunt-credentials-768x326.png.webp 768w\" sizes=\"auto, (max-width: 852px) 100vw, 852px\"><\/figure>\n<\/div>\n<p>You\u2019ll just need to name your project and select the role. <\/p>\n<p>As I\u2019m the only one with access to my projects and machine, I just set it as owner. You may want to choose otherwise. You can find out more about the roles <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/orca.security\/resources\/blog\/lateral-movement-google-cloud-default-service-account\/\">here<\/a>.<\/p>\n<p>Once you\u2019ve created the service account, you need to create a key for it. If you\u2019re not automatically taken to do so, simply click on the service account you just created:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1443\" height=\"664\" alt=\"Google Cloud - Newly created service acount\" class=\"wp-image-448396\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount.png.webp 1443w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount-600x276.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount-800x368.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount-200x92.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount-768x353.png.webp 768w\" data-lazy-sizes=\"(max-width: 1443px) 100vw, 1443px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1443\" height=\"664\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount.png.webp\" alt=\"Google Cloud - Newly created service acount\" class=\"wp-image-448396\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount.png.webp 1443w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount-600x276.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount-800x368.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount-200x92.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Newly-created-service-acount-768x353.png.webp 768w\" sizes=\"auto, (max-width: 1443px) 100vw, 1443px\"><\/figure>\n<\/div>\n<p>Click <em>Keys<\/em> in the top tabs and then <em>Create new key<\/em>.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1205\" height=\"620\" alt=\"Google Cloud - Create new key\" class=\"wp-image-448398\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key.png.webp 1205w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key-600x309.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key-800x412.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key-200x103.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key-768x395.png.webp 768w\" data-lazy-sizes=\"(max-width: 1205px) 100vw, 1205px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1205\" height=\"620\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key.png.webp\" alt=\"Google Cloud - Create new key\" class=\"wp-image-448398\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key.png.webp 1205w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key-600x309.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key-800x412.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key-200x103.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-new-key-768x395.png.webp 768w\" sizes=\"auto, (max-width: 1205px) 100vw, 1205px\"><\/figure>\n<\/div>\n<p>Choose <em>JSON<\/em>.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"561\" height=\"412\" alt=\"Google Cloud - Create private key\" class=\"wp-image-448399\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-private-key.png.webp 561w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-private-key-460x338.png.webp 460w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-private-key-154x113.png.webp 154w\" data-lazy-sizes=\"(max-width: 561px) 100vw, 561px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-private-key.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"561\" height=\"412\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-private-key.png.webp\" alt=\"Google Cloud - Create private key\" class=\"wp-image-448399\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-private-key.png.webp 561w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-private-key-460x338.png.webp 460w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Google-Cloud-Create-private-key-154x113.png.webp 154w\" sizes=\"auto, (max-width: 561px) 100vw, 561px\"><\/figure>\n<\/div>\n<p>The key will automatically download to your machine. Save it to a location where it can stay and copy the path to it. You\u2019ll need that path and the filename shortly.<\/p>\n<p>We\u2019re ready to get going! Open Jupyter Notebook and add the code blocks. I like to add short descriptions of what the blocks do for my future reference, but that\u2019s up to you.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-step-2-defining-the-article-you-want-an-outline-for\"><span class=\"ez-toc-section\" id=\"Step_2_Defining_the_article_you_want_an_outline_for\"><\/span>Step 2: Defining the article you want an outline for<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>After you have launched your Jupyter Notebook (or preferred IDE) by opening the Anaconda prompt:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"424\" height=\"195\" alt=\"Anaconda prompt\" class=\"wp-image-448400\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt.png.webp 424w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt-200x92.png.webp 200w\" data-lazy-sizes=\"(max-width: 424px) 100vw, 424px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"424\" height=\"195\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt.png.webp\" alt=\"Anaconda prompt\" class=\"wp-image-448400\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt.png.webp 424w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt-200x92.png.webp 200w\" sizes=\"auto, (max-width: 424px) 100vw, 424px\"><\/figure>\n<\/div>\n<p>And entering \u201cjupyter notebook\u201d:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"550\" height=\"144\" alt=\"Anaconda prompt - jupyter notebook\" class=\"wp-image-448401\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt-jupyter-notebook.png.webp 550w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt-jupyter-notebook-200x52.png.webp 200w\" data-lazy-sizes=\"(max-width: 550px) 100vw, 550px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt-jupyter-notebook.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"550\" height=\"144\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt-jupyter-notebook.png.webp\" alt=\"Anaconda prompt - jupyter notebook\" class=\"wp-image-448401\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt-jupyter-notebook.png.webp 550w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Anaconda-prompt-jupyter-notebook-200x52.png.webp 200w\" sizes=\"auto, (max-width: 550px) 100vw, 550px\"><\/figure>\n<\/div>\n<p>You\u2019ll create a new Python 3 notebook: <em>File &gt; New &gt; Notebook<\/em><\/p>\n<p>And start copying and pasting the code below into the cells:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"603\" height=\"198\" alt=\"Jupyter notebook - code input\" class=\"wp-image-448403\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Jupyter-notebook-code-input.png.webp 603w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Jupyter-notebook-code-input-600x197.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Jupyter-notebook-code-input-200x66.png.webp 200w\" data-lazy-sizes=\"(max-width: 603px) 100vw, 603px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Jupyter-notebook-code-input.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"603\" height=\"198\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Jupyter-notebook-code-input.png.webp\" alt=\"Jupyter notebook - code input\" class=\"wp-image-448403\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Jupyter-notebook-code-input.png.webp 603w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Jupyter-notebook-code-input-600x197.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Jupyter-notebook-code-input-200x66.png.webp 200w\" sizes=\"auto, (max-width: 603px) 100vw, 603px\"><\/figure>\n<\/div>\n<p>In the first cell, we\u2019ll enter:<\/p>\n<pre class=\"wp-block-code\"><code># Step 1: Define your target keywords and article type\n\n# Define the primary term that you're trying to rank for.\nquery = input(\"What do you want to rank for: \")\n\n# Define any secondary terms you're trying to rank for.\nquery_secondary = input(\"Are there other terms you're trying to rank for (comma separated): \")\n\n# Define the type of article outline you want to create.\narticle_type = input(\"What type of article is it (e.g., deep dive, quickstart, tutorial, etc.): \")<\/code><\/pre>\n<p>When you click this cell, it will ask you the three questions above, which will be used further below.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"982\" height=\"237\" alt=\"Step 2 output\" class=\"wp-image-448407\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output.png.webp 982w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output-600x145.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output-800x193.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output-200x48.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output-768x185.png.webp 768w\" data-lazy-sizes=\"(max-width: 982px) 100vw, 982px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"982\" height=\"237\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output.png.webp\" alt=\"Step 2 output\" class=\"wp-image-448407\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output.png.webp 982w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output-600x145.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output-800x193.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output-200x48.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-2-output-768x185.png.webp 768w\" sizes=\"auto, (max-width: 982px) 100vw, 982px\"><\/figure>\n<\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_3_Installing_the_libraries_youll_need\"><\/span>Step 3: Installing the libraries you\u2019ll need<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The next step is to install all the libraries we\u2019ll need. They are:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>google.cloud.language_v1<\/strong>: This is part of the Google Cloud NLP library, used for analyzing entities.<\/li>\n<li><strong>googleapiclient.discovery.build<\/strong>: We\u2019re using this one for our custom search engine.<\/li>\n<li><strong>openai<\/strong>: Not surprisingly, we\u2019re using this one to work with the OpenAI API and use GPT-4o.<\/li>\n<li><strong>firecrawl<\/strong>: We\u2019re using this one to scrape the top webpages.<\/li>\n<li><strong>wandb<\/strong> and <strong>weave<\/strong>: We\u2019re using these two libraries to log our prompts and other inputs for evaluation.<\/li>\n<li><strong>Some supporting libraries,<\/strong> like os, time and getpass, that simply facilitate functionality.<\/li>\n<\/ul>\n<p>You\u2019ll install these with the following code:<\/p>\n<pre class=\"wp-block-code\"><code># Step 2: Install and import the necessary libraries\n\n# Install required packages\n!pip install --upgrade google-api-python-client google-cloud-language openai weave wandb firecrawl\n\n# Import necessary libraries\nimport os\nfrom getpass import getpass\nfrom collections import defaultdict\nfrom google.cloud import language_v1\nfrom googleapiclient.discovery import build\nfrom openai import OpenAI\nimport re\nimport time\nfrom firecrawl import FirecrawlApp\n\n# Import wandb and weave for logging and visualization\nimport wandb\nimport weave<\/code><\/pre>\n<p>This might take a minute or two to run and will look like:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"994\" height=\"545\" alt=\"Step 3 output\" class=\"wp-image-448408\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output.png.webp 994w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output-600x329.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output-800x439.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output-200x110.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output-768x421.png.webp 768w\" data-lazy-sizes=\"(max-width: 994px) 100vw, 994px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"994\" height=\"545\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output.png.webp\" alt=\"Step 3 output\" class=\"wp-image-448408\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output.png.webp 994w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output-600x329.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output-800x439.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output-200x110.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-3-output-768x421.png.webp 768w\" sizes=\"auto, (max-width: 994px) 100vw, 994px\"><\/figure>\n<\/div>\n<p><!-- START INLINE FORM --><\/p>\n<p><!-- END INLINE FORM --><\/p>\n<hr class=\"wp-block-separator has-text-color has-cyan-bluish-gray-color has-css-opacity has-cyan-bluish-gray-background-color has-background\">\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_4_Adding_our_APIs\"><\/span>Step 4: Adding our APIs<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>This section differs from the Colab, where you enter your API keys one by one to avoid leaving them in a public location. <\/p>\n<p>If you are working on your own secure machine, you may want to add your API keys directly into the script.<\/p>\n<p>Alternatively, you can use the code from the Colab for this block and have the script ask you for your API keys each time.<\/p>\n<p>The code for this step is:<\/p>\n<pre class=\"wp-block-code\"><code># Google API Key\ngoogle_api = 'YOUR_GOOGLE_API_KEY'\n\n# Google Application Credentials (JSON file path)\nos.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'C:\/Users\/PATH_TO_YOUR_APPLICATION_CREDENTIALS\/FILE.JSON'\n\n# Google Custom Search Engine ID\ngoogle_search_id = 'SEARCH_ENGINE_ID'\n\n\n# Firecrawl API Key\nfirecrawl_api_key = 'FIRECRAWL_API_KEY'\napp = FirecrawlApp(api_key=firecrawl_api_key)\n\n\n# Initialize Weights &amp; Biases\nwandb_api_key = 'WEIGHTS_AND_BIASES_API_KEY'\nwandb.login(key=wandb_api_key)\n\n# OpenAI API Key\nopenai_api_key = 'OPENAI_API_KEY'\nos.environ[\"OPENAI_API_KEY\"] = openai_api_key<\/code><\/pre>\n<p>Remember above where I noted you\u2019d need to copy the path to the application file and also the filename itself. That\u2019s what you put in the block:<\/p>\n<pre class=\"wp-block-code\"><code>C:\/Users\/PATH_TO_YOUR_APPLICATION_CREDENTIALS\/FILE.JSON<\/code><\/pre>\n<p>For example, that line in my script looks like:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"696\" height=\"43\" alt=\"Step 4 sample script\" class=\"wp-image-448409\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-4-sample-script.png.webp 696w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-4-sample-script-600x37.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-4-sample-script-200x12.png.webp 200w\" data-lazy-sizes=\"(max-width: 696px) 100vw, 696px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-4-sample-script.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"696\" height=\"43\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-4-sample-script.png.webp\" alt=\"Step 4 sample script\" class=\"wp-image-448409\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-4-sample-script.png.webp 696w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-4-sample-script-600x37.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-4-sample-script-200x12.png.webp 200w\" sizes=\"auto, (max-width: 696px) 100vw, 696px\"><\/figure>\n<\/div>\n<p>We\u2019ve now set the stage. Let\u2019s get into the meat of the script.<\/p>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_5_Defining_the_functions\"><\/span>Step 5: Defining the functions<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The next step is to define the functions we\u2019ll need to make this all work.\u00a0<\/p>\n<p>The functions we\u2019re defining are:<\/p>\n<ul class=\"wp-block-list\">\n<li><strong><code>google_search<\/code><\/strong>: This function uses Google\u2019s Custom Search API to pull the top 10 results for the primary term we provided in step one.<\/li>\n<li><strong><code>fetch_content_with_firecrawl<\/code><\/strong>: This one\u2019s our scraper, fetching content from the top 10 using Firecrawl. If a page blocks access, it\u2019ll let you know and move on.<\/li>\n<li><strong><code>extract_headings_from_markdown<\/code><\/strong>: Parses markdown content and extracts the headings from the scraped content.\u00a0<\/li>\n<li><strong><code>generate_summary<\/code><\/strong>: This function leverages GPT-4o to create a concise summary of each of the top pages based in part on the headings.\u00a0<\/li>\n<li><strong><code>extract_questions<\/code><\/strong>: This one pulls out essential questions from the top-ranking pages using GPT-4o.\u00a0<\/li>\n<li><strong><code>top_questions<\/code><\/strong>: This function sorts through all the extracted questions to find the most relevant and useful ones across all the top-ranking pages.<\/li>\n<li><strong><code>analyze_entities<\/code><\/strong>: This one uses Google Cloud NLP to analyze and extract entities (the \u201cwho,\u201d \u201cwhat,\u201d and \u201cwhere\u201d of the content). This helps us understand what concepts are central to the text, so we can make sure these core elements are woven into the article outline.<\/li>\n<\/ul>\n<p>The code for this is:<\/p>\n<pre class=\"wp-block-code\"><code># Step 4: Define functions\n\n# Setup Google Search API\ndef google_search(search_term, api_key, cse_id, **kwargs):\n    service = build(\"customsearch\", \"v1\", developerKey=api_key)\n    res = service.cse().list(q=search_term, cx=cse_id, **kwargs).execute()\n    return res['items']\n\n# Function to extract content from a webpage using Firecrawl\ndef fetch_content_with_firecrawl(url):\n    try:\n        scrape_result = app.scrape_url(url, params={'formats': ['markdown']})\n        if '403 Forbidden' in scrape_result.get('status', ''):\n            print(f\"Access to {url} was denied with a 403 Forbidden error.\")\n            return None\n        page_text = scrape_result.get('markdown', '')\n        if not page_text:\n            print(f\"No content available for {url}\")\n            return None\n        return page_text\n    except Exception as e:\n        print(f\"Error fetching content from {url}: {str(e)}\")\n        return None\n\n# Function to extract headings from markdown text\ndef extract_headings_from_markdown(markdown_text):\n    \"\"\"Extract headings from markdown text based on markdown syntax.\"\"\"\n    headings = []\n    for line in markdown_text.split('\\n'):\n        line = line.s<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/trip-and-travel\/\" data-internallinksmanager029f6b8e52c=\"10\" title=\"Trip &amp; Travel\" target=\"_blank\" rel=\"noopener\">trip<\/a>()\n        if line.startswith('#'):\n            # Remove leading '#' characters and any extra whitespace\n            heading = line.lstrip('#').strip()\n            if heading:\n                headings.append(heading)\n    return headings\n\n# Function to generate a summary of the text using OpenAI GPT-4o\ndef generate_summary(text, headings):\n    \"\"\"Generate a GPT-4o summary of the text using the headings.\"\"\"\n    # Prepare the prompt\n    headings_text=\"\\n\".join(f\"- {heading}\" for heading in headings)\n    prompt = (f\"Summarize the following article, focusing on these headings:\\n{headings_text}\\n\\n\"\n              f\"The summary should be concise (max 500 tokens) and capture the key points.\")\n    try:\n        response = client.chat.completions.create(\n            messages=[\n                {\"role\": \"system\", \"content\": \"You are an expert summarizer.\"},\n                {\"role\": \"user\", \"content\": prompt + \"\\n\\n\" + text}\n            ],\n            model=\"gpt-4o\",\n            max_tokens=500,\n            temperature=0.2,\n            n=1\n        )\n        summary = response.choices[0].message.content.strip()\n        return summary\n    except Exception as e:\n        print(f\"Error generating summary: {e}\")\n        return \"Summary not available.\"\n\n# Function to extract questions from the text using OpenAI GPT-4o\ndef extract_questions(text):\n    \"\"\"Extract questions from the text using GPT-4o.\"\"\"\n    prompt = (f\"Extract the top 5 most important questions from the following text related to the query '{query}'. \"\n              f\"List them as bullet points.\\n\\n{text}\")\n    try:\n        response = client.chat.completions.create(\n            messages=[\n                {\"role\": \"system\", \"content\": \"You are a helpful assistant who extracts key questions from texts.\"},\n                {\"role\": \"user\", \"content\": prompt}\n            ],\n            model=\"gpt-4o\",\n            max_tokens=1000,\n            temperature=0.1,\n            n=1\n        )\n        questions_text = response.choices[0].message.content.strip()\n        # Split the response into individual questions based on bullet points\n        questions = re.findall(r\"-\\s*(.*)\", questions_text)\n        if not questions:\n            questions = [questions_text]\n        return questions\n    except Exception as e:\n        print(f\"Error extracting questions: {e}\")\n        return []\n\n# Function to select the top questions\ndef top_questions(all_questions):\n    \"\"\"Generate the top questions from the list of all questions.\"\"\"\n    try:\n        questions_text=\"\\n\".join(f\"- {question}\" for question in all_questions)\n        prompt = (f\"From the following list of questions extracted from top articles about '{query}', \"\n                  f\"select the 5 most important questions that would be most useful to the user. \"\n                  f\"List them as bullet points.\\n\\n{questions_text}\")\n        response = client.chat.completions.create(\n            messages=[\n                {\"role\": \"system\", \"content\": \"You are an expert at identifying key questions on a topic.\"},\n                {\"role\": \"user\", \"content\": prompt}\n            ],\n            model=\"gpt-4o\",\n            max_tokens=500,\n            temperature=0.1,\n            n=1\n        )\n        top_questions_text = response.choices[0].message.content.strip()\n        # Split the response into individual questions based on bullet points\n        top_questions_list = re.findall(r\"-\\s*(.*)\", top_questions_text)\n        if not top_questions_list:\n            top_questions_list = [top_questions_text]\n        return top_questions_list\n    except Exception as e:\n        print(f\"Error generating top questions: {e}\")\n        return []\n\n# Function to analyze entities using Google Cloud NLP\ndef analyze_entities(text_content):\n    \"\"\"Analyze entities in the text using Google Cloud NLP.\"\"\"\n    try:\n        document = language_v1.Document(content=text_content, type_=language_v1.Document.Type.PLAIN_TEXT)\n        response = nlp_client.analyze_entities(document=document, encoding_type=language_v1.EncodingType.UTF8)\n        return response.entities\n    except Exception as e:\n        print(f\"Error analyzing entities: {e}\")\n        return []<\/code><\/pre>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_6_Scraping_the_top_10_and_collecting_the_information\"><\/span>Step 6: Scraping the top 10 and collecting the information<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>We\u2019re almost there.\u00a0In this step, we use the functions above and send our crawler across the top 10 ranking pages (you can change this number).<\/p>\n<p>We\u2019re also going to write some of the core information to Weights &amp; Biases so we can review the various inputs we\u2019ve collected.\u00a0<\/p>\n<p>When I refer to inputs, I mean the entities, questions and summaries we\u2019ll use to augment our article outline generation prompt in step seven, our final step.<\/p>\n<p>The code for this step is:<\/p>\n<pre class=\"wp-block-code\"><code># Step 5: Scrape and analyze the top ranking pages\n\nclient = OpenAI()\n\n# Initialize Weights &amp; Biases\nwandb.init(project=\"wandb-article-outlines\")\nweave.init('wandb-article-outlines')\n\n# Create W&amp;B Tables to store scraped data\nfirecrawl_table = wandb.Table(columns=[\n    \"url\",\n    \"markdown_summary\",\n    \"artifact_link\",\n    \"title\",\n    \"description\",\n    \"language\",\n    \"status_code\"\n])\ntop_questions_table = wandb.Table(columns=[\n    \"question\"\n])\n\nentities_table = wandb.Table(columns=[\n    \"entity\",\n    \"aggregated_score\",\n    \"page_count\"\n])\n\n# Initialize a list to collect all questions, entities and summaries\nall_questions = []\nentity_data = {}\nmarkdown_summaries = []\n\n# Initialize Google Cloud NLP client\nnlp_client = language_v1.LanguageServiceClient()\n\n# Search and scrape top 10 pages\nsearch_results = google_search(query, google_api, google_search_id, num=10)\n\nfor result in search_results:\n    url = result['link']\n    print(f\"Processing URL: {url}\")\n    # Fetch content using Firecrawl\n    page_text = fetch_content_with_firecrawl(url)\n    if page_text is None:\n        print(f\"Failed to fetch content from {url}\")\n        continue  # Skip if no content\n\n    # Save the full content as a file\n    safe_title=\"\".join(c if c.isalnum() else '_' for c in result.get('title', 'page_text'))\n    artifact_filename = f\"{safe_title}.txt\"\n    with open(artifact_filename, 'w', encoding='utf-8') as f:\n        f.write(page_text)\n\n    # Create and log the artifact\n    artifact = wandb.Artifact(name=f\"page_text_{safe_title}\", type=\"page_text\")\n    artifact.add_file(artifact_filename)\n    artifact = wandb.run.log_artifact(artifact)  # Capture the logged artifact\n    artifact.wait()\n    artifact_link = artifact.get_entry(artifact_filename).ref_url\n\n    # Extract metadata\n    title = result.get('title', 'Unknown Title')\n    description = result.get('snippet', 'No description available')\n    language=\"en\"\n    status_code = 200 \n\n    # Extract headings from the markdown text\n    headings = extract_headings_from_markdown(page_text)\n\n    # Generate a summary using GPT-4\n    markdown_summary = generate_summary(page_text, headings)\n    if markdown_summary:\n        markdown_summaries.append(markdown_summary)\n    else:\n        print(f\"No summary generated for {url}\")\n\n    # Extract questions from the page and add them to the list\n    questions = extract_questions(page_text)\n    all_questions.extend(questions)\n\n    # Analyze entities in the page text\n    entities = analyze_entities(page_text)\n    page_entities = set()  # To track unique entities in this page\n\n    for entity in entities:\n        entity_name = entity.name\n        salience = entity.salience\n        # Update entity data\n        if entity_name in entity_data:\n            entity_info = entity_data[entity_name]\n            entity_info['total_salience'] += salience\n            if url not in entity_info['pages']:\n                entity_info['page_count'] += 1\n                entity_info['pages'].add(url)\n        else:\n            entity_data[entity_name] = {\n                'total_salience': salience,\n                'page_count': 1,\n                'pages': {url}\n            }\n\n    # Add data to the table, including the markdown summary and artifact link\n    firecrawl_table.add_data(\n        url,\n        markdown_summary,\n        artifact_link,\n        title,\n        description,\n        language,\n        status_code\n    )\n\n    # Clean up the temporary file\n    os.remove(artifact_filename)\n\n# After processing all pages, generate the top questions\ntop_questions_list = top_questions(all_questions)\n\n# Add the top questions to the table\nfor question in top_questions_list:\n    top_questions_table.add_data(question)\n\n# Determine the top entities\n# Calculate a combined score: total_salience * page_count\nfor entity_name, data in entity_data.items():\n    aggregated_score = data['total_salience'] * data['page_count']\n    data['aggregated_score'] = aggregated_score\n\n# Sort entities by the aggregated score\ntop_entities = sorted(entity_data.items(), key=lambda item: item[1]['aggregated_score'], reverse=True)\n\n# Get the top N entities (e.g., top 10)\ntop_n = 10\ntop_entities = top_entities[:top_n]\n\n# Add top entities to the entities table\nfor entity_name, data in top_entities:\n    entities_table.add_data(\n        entity_name,\n        data['aggregated_score'],\n        data['page_count']\n    )\n\n# Log the tables to W&amp;B\nwandb.log({\n    \"scraped_data\": firecrawl_table,\n    \"top_questions_table\": top_questions_table,\n    \"entities_table\": entities_table,\n    \"markdown_summaries\": markdown_summaries,\n})\n\nprint(\"Markdown Summaries:\", markdown_summaries)\n\n# Finish the W&amp;B run\nwandb.finish()<\/code><\/pre>\n<p>The cell will take a few minutes to finish running, and when it does, you\u2019ll see something like:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"354\" alt=\"Step 6 output\" class=\"wp-image-448410\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-800x354.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-600x266.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-200x89.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-768x340.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output.png.webp 955w\" data-lazy-sizes=\"(max-width: 800px) 100vw, 800px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-800x354.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"354\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-800x354.png.webp\" alt=\"Step 6 output\" class=\"wp-image-448410\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-800x354.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-600x266.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-200x89.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-768x340.png.webp 768w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output.png.webp 955w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\"><\/figure>\n<\/div>\n<p>The system displays entities, questions, and summaries directly on the screen after a run. <\/p>\n<p>However, I prefer a simpler approach: clicking the <em>Last run<\/em> link to view the results instead.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1045\" height=\"639\" alt=\"Step 6 output A\" class=\"wp-image-448413\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1.png.webp 1045w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1-553x338.png.webp 553w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1-800x489.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1-185x113.png.webp 185w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1-768x470.png.webp 768w\" data-lazy-sizes=\"(max-width: 1045px) 100vw, 1045px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1045\" height=\"639\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1.png.webp\" alt=\"Step 6 output A\" class=\"wp-image-448413\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1.png.webp 1045w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1-553x338.png.webp 553w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1-800x489.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1-185x113.png.webp 185w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-A-1-768x470.png.webp 768w\" sizes=\"auto, (max-width: 1045px) 100vw, 1045px\"><\/figure>\n<\/div>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1057\" height=\"330\" alt=\"Step 6 output B\" class=\"wp-image-448414\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B.png.webp 1057w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B-600x187.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B-800x250.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B-200x62.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B-768x240.png.webp 768w\" data-lazy-sizes=\"(max-width: 1057px) 100vw, 1057px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"1057\" height=\"330\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B.png.webp\" alt=\"Step 6 output B\" class=\"wp-image-448414\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B.png.webp 1057w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B-600x187.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B-800x250.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B-200x62.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-6-output-B-768x240.png.webp 768w\" sizes=\"auto, (max-width: 1057px) 100vw, 1057px\"><\/figure>\n<\/div>\n<p>Here, we can review the output being passed to the prompt. This can help identify where things may go awry if you\u2019re finding your outlines aren\u2019t quite right.<\/p>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_7_Generating_our_article_outline\"><\/span>Step 7: Generating our article outline<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Now we can generate the article outline.<\/p>\n<p>I invite you to review and edit the prompt to suit the formats you\u2019ve found helpful. <\/p>\n<p>The one here is fairly generic. It is based on formats and rules I\u2019ve found successful, but with some specifics to individual use cases removed.<\/p>\n<p>The code to create the outline is:<\/p>\n<pre class=\"wp-block-code\"><code># Generate the article outline using the collected data\n@weave.op()\ndef generate_outline(top_entities, top_questions, query, query_secondary, article_type, markdown_summaries):\n    entities_str=\", \".join([entity_name for entity_name, _ in top_entities])\n    questions_str=\", \".join(top_questions)\n    summaries_str=\"\\n\\n\".join(markdown_summaries)\n    try:\n        response = client.chat.completions.create(\n                model=\"gpt-4o\",\n                messages=[\n                    {\"role\": \"system\",\n                     \"content\": \"You create succinct and clear article outlines. You can include your understanding \"\n                       \"of a topic to enhance an outline, but the focus should be on the inclusion of the entities, questions and top ranking content you are provided with.\"},\n                    {\"role\": \"assistant\", \"content\": \"You are a highly skilled writer and you want to produce a \" + article_type +\n                       \" article outline that will appeal to users and rank well for queries given by the user. \"\n                       \"The outline will contain headings and sub-headings, with clear and concise descriptions of the content \"\n                       \"that is recommended for that section and why. \\n When you are recommending to create an introductory paragraph to a section, to capture a featured snippet, \"\n                        \"note that it should be between 260 and 320 characters and then provide a clearly noted example of what one might be. \"\n                         \"\\n After you have provided the outline, explain clearly how this article outline \"\n                       \"could be used to create an article that will rank well using best-practice SEO strategies as well as be helpful \"\n                       \"to users. You will be judged based on how well the article ranks, as well as how engaging the article is to readers, \"\n                       \"and provide the metrics you would suggest be used to judge whether you are successful. \\n An example of an article \"\n                       \"structure that works well is: \\n\\n\"\n                       \"Title: Top-Level Content (e.g., An Introduction to [Main Topic])\\n\\n\"\n                       \"The description: Visible on the page as well as used as the description to Google. Should be 130 &lt;= character_count \"\n                       \"&lt;= 160 and include the main keywords whenever possible.\\n\\n\"\n                       \"**Introduction**\\n\\n\"\n                       \"We don't need a heading tag here. Simply dive in with a brief description of what you'll be covering. One or two short \"\n                       \"paragraphs is great, but longer is fine.\\n\\n\"\n                       \"**H2 - Table Of Contents**\\n\"\n                       \"Ideally this section is done manually, but in a pinch, you can use the \/ Table Of Contents feature. You can add a bit of \"\n                       \"additional content below the table of contents if you like.\\n\\n\"\n                       \"**H2 - What Is [Main Topic]?**\\n\"\n                       \"Ideally here we have a section on the main topic. It will start with a paragraph that runs between 230 and 260 characters. \"\n                       \"This first paragraph should be the short answer to the question and it will hopefully get picked up as a featured snippet \"\n                       \"and be used for voice search. It can and should be extended from there to provide additional information and context.\\n\\n\"\n                       \"**H2 - Optional - Subset Such As 'Types Of [Main Topic]'**\\n\"\n                       \"Brief description here--ideally 230 to 260 characters.\\n\\n\"\n                       \"**H3 - [Subset Type One] (e.g., First Subtype of [Main Topic])**\\n\"\n                       \"Description of [Subset Type One]. Ideally starting with 260 to 320 characters, but not mandatory and expanding from there.\\n\\n\"\n                       \"**H3 - [Subset Type Two] (e.g., Second Subtype of [Main Topic])**\\n\"\n                       \"Description of [Subset Type Two]. Ideally starting with 260 to 320 characters, but not mandatory and expanding from there.\\n\\n\"\n                        \"**H2 - A tutorial for [Main Topic]**\\n\"\n                       \"Generate a tutorial based on common macnmachine learning tasks, which are likely found in the summaries provided by the user. You should use W&amp;B Weave whenever possible.\\n\\n\"\n                       \"**H2 - What Is [Main Topic] Used For?**\\n\"\n                       \"Again, ideally this starts with a 230 to 260 character short answer and is expanded upon.\\n\\n\"\n                       \"**H2 - Examples Of [Main Topic]** \\n\"\n                       \"Optionally, we can place a brief description of the types of examples. It should be done in H3 tags (assuming it's a simple one). \"\n                       \"A robust example requiring multiple stages (e.g., setup, running, visualizing) may require multiple H2 tags with H3s nested beneath.\\n\"\n                       \"**H2 - Recommended Reading On [Main Topic]** \\n\"\n                       \"Here we simply add a list with 2 or 4 articles you feel are related and would be of interest to the reader.\"},\n                    {\"role\": \"user\",\n            \"content\": \"Create an article outline that will rank well for \" + query + \" as the primary term and \" + query_secondary +\n                       \" secondary keywords, which are less important but should still be considered. The following entities appear to be \"\n                       \"relevant to ranking in the top 10 and should be worked into the page:\\n\" + entities_str + \"\\n Try to ensure the outline \"\n                       \"will make it easy to work these into the article prominently and explain how this might be done in comments. Additionally, \"\n                       \"the following questions appear to be important to answer in the article:\\n\" + questions_str +\n                       \"\\n The following are summaries of the content and format that can be found on the top-ranking pages. This should heavily influence \"\n                       \"the outlines you produce, as this content ranks well: \\n\" + summaries_str + \"\\n\"\n                       \"Try to ensure that it will be easy to answer these questions in the article and again, explain how you would recommend \"\n                       \"doing this in a way that will seem useful to the user. The article outline should begin by explaining \\n- all of the core \"\n                       \"concepts required to understand the topic\"\n        }],\n                max_tokens=2000,\n                temperature=0.2,\n                n=1\n            )\n        return response.choices[0].message.content.strip()\n    except Exception as e:\n        print(f\"Error generating outline: {e}\")\n        return \"Outline not available.\"\n\n# Generate the article outline\narticle_outline = generate_outline(\n    top_entities,\n    top_questions_list,\n    query,\n    query_secondary,\n    article_type,\n    markdown_summaries\n)\n\n# Optionally, you can print or save the article outline\nprint(\"Generated Article Outline:\")\nprint(article_outline)<\/code><\/pre>\n<p>You\u2019ll get your outline once you\u2019ve run this (about a minute or so).<\/p>\n<p>If you look at the prompts, you\u2019ll see that I\u2019ve told it to develop the article outline using the article type we defined in Step 1, as well as the targeted queries, top question, entities and summaries.<\/p>\n<p>As noted above, this allows us to generate articles inspired by what\u2019s working on the best-performing sites.<\/p>\n<p>The default output looks like this:<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"970\" height=\"362\" alt=\"Step 7 output\" class=\"wp-image-448415\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output.png.webp 970w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output-600x224.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output-800x299.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output-200x75.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output-768x287.png.webp 768w\" data-lazy-sizes=\"(max-width: 970px) 100vw, 970px\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output.png.webp\"><img loading=\"lazy\" decoding=\"async\" width=\"970\" height=\"362\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output.png.webp\" alt=\"Step 7 output\" class=\"wp-image-448415\" srcset=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output.png.webp 970w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output-600x224.png.webp 600w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output-800x299.png.webp 800w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output-200x75.png.webp 200w,https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/Step-7-output-768x287.png.webp 768w\" sizes=\"auto, (max-width: 970px) 100vw, 970px\"><\/figure>\n<\/div>\n<p>But if we click the link at the top, we are moved into W&amp;B Weave, where we can contextualize that outline.\u00a0<\/p>\n<p>The GIF below illustrates how we can review the entities and their salience scores, top questions, URL summaries, queries and article types alongside the output (the outline) for troubleshooting and to understand the information used to generate the outline.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"799\" height=\"413\" alt=\"How to create article outlines based on key traits from Google\u2019s Top 10\" class=\"wp-image-448416\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/How-to-create-article-outlines-based-on-key-traits-from-Googles-Top-10.gif\"><img loading=\"lazy\" decoding=\"async\" width=\"799\" height=\"413\" src=\"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/How-to-create-article-outlines-based-on-key-traits-from-Googles-Top-10.gif\" alt=\"How to create article outlines based on key traits from Google\u2019s Top 10\" class=\"wp-image-448416\"><\/figure>\n<\/div>\n<h2 class=\"wp-block-heading\" id=\"h-just-the-beginning\"><span class=\"ez-toc-section\" id=\"Just_the_beginning\"><\/span>Just the beginning<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Hopefully you find this script and its features as useful as I do. I also hope you don\u2019t just use the script without making any changes. <\/p>\n<p>Explore how to adjust the prompts, add to them and maybe even think about different APIs you could supplement the data with.<\/p>\n<p>I\u2019m already working on some new enhancements. I\u2019d love to hear your ideas.<\/p>\n<\/div>\n<p><\/p>\n<div class=\"about-author\">\n<p>About the author<\/p>\n<div class=\"information\">\n<div class=\"author-module\">\n<div class=\"row\">\n<div class=\"col-12 col-lg-3 text-center\">\n<div class=\"avatar\">\n\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" class=\"img-fluid rounded-circle avatar-border\" alt=\"Dave Davies\" width=\"140\" height=\"140\" src=\"https:\/\/searchengineland.com\/images\/authors\/DaveDavies-lg.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"img-fluid rounded-circle avatar-border\" src=\"https:\/\/searchengineland.com\/images\/authors\/DaveDavies-lg.jpg\" alt=\"Dave Davies\" width=\"140\" height=\"140\">\n\t\t\t\t\t\t\t\t\t\t\t<\/div>\n<\/p><\/div>\n<div class=\"col-12 col-lg-9\">\n<div class=\"about\">\n<div class=\"name\">\n\t\t\t\t\t\t\t<strong>Dave Davies<\/strong>\n\t\t\t\t\t\t<\/div>\n<div class=\"row g-2 pt-2\">\n<div class=\"col-auto twitter\">\n\t\t\t\t\t\t\t\t\t<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/twitter.com\/intent\/follow?original_referer=https%3A%2F%2Fsearchengineland.com%2F&amp;region=follow_link&amp;screen_name=beanstalkim&amp;tw_p=followbutton&amp;variant=2.0\" rel=\"me\" target=\"_blank\" aria-label=\"opens in a new tab\"><i class=\"fab fa-x-twitter\"><\/i><\/a>\n\t\t\t\t\t\t\t<\/div>\n<div class=\"col-auto\">\n\t\t\t\t\t\t\t\t\t<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/in\/daverdavies\/?originalSubdomain=ca\" target=\"_blank\" aria-label=\"opens in a new tab\"><i class=\"fab fa-linkedin\"><\/i><\/a>\n\t\t\t\t\t\t\t\t<\/div>\n<\/p><\/div>\n<p>\t\t\t\t\t\tDave Davies is the Lead SEO for the Machine Learning Operations company Weights &amp; Biases. He got his start in SEO in the early 2000s and in 2004 co-founded Beanstalk Internet Marketing with his wife Mary, who still runs its day-to-day operations. He hosts a weekly podcast, speaks regularly at the industry&#8217;s leading conferences, and is proud to be a regular contributor right here on Search Engine Land.\t\t\t\t\t<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" >Technology<\/a><\/span> category.<\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/searchengineland.com\/create-article-outlines-ai-google-top-10-448385\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Use this guide and script to automate SEO content research to generate article outlines using AI, APIs and top-ranking page analysis. The world of SEO has gotten increasingly nuanced. If you\u2019re here, you know that merely writing an article and buying links to it hasn\u2019t worked in a long while.\u00a0 When I\u2019m working on my&#8230;<\/p>\n","protected":false},"author":1,"featured_media":644669,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/searchengineland.com\/wp-content\/seloads\/2024\/11\/How-to-create-article-outlines-based-on-key-traits-from-Googles-top-10.png","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[78070],"class_list":["post-644668","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology","tag-seo"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/644668","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=644668"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/644668\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/644669"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=644668"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=644668"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=644668"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}