{"id":492519,"date":"2022-09-13T03:48:56","date_gmt":"2022-09-13T00:48:56","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/what-happens-during-a-kubernetes-control-plane-failure\/"},"modified":"2022-09-13T03:48:56","modified_gmt":"2022-09-13T00:48:56","slug":"what-happens-during-a-kubernetes-control-plane-failure","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/what-happens-during-a-kubernetes-control-plane-failure\/","title":{"rendered":"#What Happens During a Kubernetes Control Plane Failure?"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a39460f75e24\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a39460f75e24\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/what-happens-during-a-kubernetes-control-plane-failure\/#%E2%80%9CWhat_Happens_During_a_Kubernetes_Control_Plane_Failure%E2%80%9D\" >&#8220;What Happens During a Kubernetes Control Plane Failure?&#8221;<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/what-happens-during-a-kubernetes-control-plane-failure\/#Understanding_the_Control_Plane\" >Understanding the Control Plane<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/what-happens-during-a-kubernetes-control-plane-failure\/#What_Happens_During_Control_Plane_Failure\" >What Happens During Control Plane Failure?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/what-happens-during-a-kubernetes-control-plane-failure\/#What_About_Worker_Nodes_and_Running_Pods\" >What About Worker Nodes and Running Pods?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/what-happens-during-a-kubernetes-control-plane-failure\/#Avoiding_Control_Plane_Failure\" >Avoiding Control Plane Failure<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/buradabiliyorum.com\/en\/what-happens-during-a-kubernetes-control-plane-failure\/#Summary\" >Summary<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"%E2%80%9CWhat_Happens_During_a_Kubernetes_Control_Plane_Failure%E2%80%9D\"><\/span>&#8220;What Happens During a Kubernetes Control Plane Failure?&#8221;<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<div>\n<img loading=\"lazy\" decoding=\"async\" class=\"type:primaryImage alignnone size-full wp-image-806255\" data-pagespeed-no-defer=\"\" src=\"https:\/\/www.howtogeek.com\/wp-content\/uploads\/2022\/05\/Kubernetes-New.jpg?width=1198&amp;trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Graphic with the Kubernetes logo\" width=\"1202\" height=\"677\"\/><\/p>\n<p>Kubernetes is the leading orchestrator for distributing container instances across multiple physical nodes. The nodes are managed by the Kubernetes control plane, a <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/overview\/components\">collection of components<\/a> which maintain the cluster\u2019s state, respond to changing conditions, and handle scheduling decisions.<\/p>\n<p>It\u2019s essential to understand the control plane\u2019s role when you\u2019re operating clusters that need consistent availability. In this article, you\u2019ll learn what h<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>ens when the control plane fails so you can plan ahead and implement protections.<\/p>\n<h2 id=\"understanding-the-control-plane\"><span class=\"ez-toc-section\" id=\"Understanding_the_Control_Plane\"><\/span>Understanding the Control Plane<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The Kubernetes control plane is responsible for your cluster\u2019s global operations. It coordinates actions that affect your worker nodes. The control plane also provides <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/tasks\/administer-cluster\/configure-upgrade-etcd\">etcd data storage<\/a> for the cluster, as well as the API server which you interact with using tools like Kubectl.<\/p>\n<p>Here are some of the control plane\u2019s main responsibilities:<\/p>\n<ul>\n<li><strong>kube-apiserver<\/strong> hosts the Kubernetes API server.<\/li>\n<li><strong>kube-controller-manager<\/strong> starts and runs the controllers within your cluster, allowing state changes requested by the API server to be detected and applied.<\/li>\n<li><strong>kube-scheduler<\/strong> assigns Pods to worker nodes by determining which node is best equipped to support each new Pod.<\/li>\n<li><strong>etcd<\/strong> is a key-value data store that holds all Kubernetes cluster data and state information.<\/li>\n<\/ul>\n<p>The Kubernetes architecture relies on these components being continually available. They work together to create the normal operating state where everything runs smoothly and your cluster meets expectations.<\/p>\n<h2 id=\"what-happens-during-control-plane-failure\"><span class=\"ez-toc-section\" id=\"What_Happens_During_Control_Plane_Failure\"><\/span>What Happens During Control Plane Failure?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The control plane is not impervious to failure. The high number of components involved means individual pieces can stop working and cause knock-on effects in your cluster. A component might crash or the physical host running the control plane could suffer a hardware failure.<\/p>\n<p>The actual effects on your cluster will vary depending on the component with the problem. However a control plane failure will usually prevent you from administering your cluster and could stop existing workloads from reacting to new events:<\/p>\n<ul>\n<li>If the API server fails, Kubectl, the Kubernetes dashboard, and other management tools will stop working.<\/li>\n<li>If the scheduler fails, new Pods won\u2019t get allocated to nodes so they\u2019ll be inaccessible and show as stuck in the Pending state. This will also affect Pods that need to be rescheduled because a Node\u2019s run out of resources or a scaling request has been sent.<\/li>\n<li>When the controller manager fails, changes you apply to your cluster won\u2019t be picked up, so your workloads will appear to retain their previous state.<\/li>\n<\/ul>\n<p>Control plane failures prevent you from effectively modifying cluster state. Changes will either fail altogether or have no effect inside the cluster.<\/p>\n<h2 id=\"what-about-worker-nodes-and-running-pods\"><span class=\"ez-toc-section\" id=\"What_About_Worker_Nodes_and_Running_Pods\"><\/span>What About Worker Nodes and Running Pods?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The control plane is a management layer that sits above and spans across your worker nodes and their Pods. The control plane and the workers are independent of each other. Once a Pod\u2019s been scheduled to a node, that node becomes responsible for acquiring the correct image and running a container instance.<\/p>\n<p>This means failures in the control plane won\u2019t necessarily knock out workloads that are already in a healthy state. You can often continue accessing existing Pods, even when you can\u2019t connect to your cluster with Kubectl. Users won\u2019t necessarily notice a short-term control plane outage.<\/p>\n<p>Longer periods of downtime increase the probability that worker nodes will start to face issues too. Nodes won\u2019t be able to reconcile their state so inconsistencies could occur. Networking can also start to break up if DNS isn\u2019t working and the contents of cached requests expire.<\/p>\n<p>A failure can become more serious if a worker node starts to experience problems while the control plane is down. In this situation Pods on the node may stop running but the rest of the cluster will be oblivious to what\u2019s happening. It\u2019ll be impossible to reschedule the Pods to another node as nodes operate independently in the control plane\u2019s absence. This will cause your workload to drop offline.<\/p>\n<h2 id=\"avoiding-control-plane-failure\"><span class=\"ez-toc-section\" id=\"Avoiding_Control_Plane_Failure\"><\/span>Avoiding Control Plane Failure<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>You can defend against control plane failure by setting up a highly available cluster that replicates control plane functions across several machines. In the same way you use Kubernetes to distribute and scale your own containers, you can apply high availability (HA) to Kubernetes itself to increase resiliency.<\/p>\n<p>Kubernetes <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/setup\/production-environment\/tools\/kubeadm\/high-availability\/#first-steps-for-both-methods\">offers two mechanisms<\/a> for setting up an HA control plane implementation:<\/p>\n<ol type=\"1\">\n<li><strong>Using \u201cstacked\u201d control plane nodes.<\/strong> \u2013 This approach requires less infrastructure and works with a minimum of three machines. Each machine will run its own control plane that replicates data from the others. One host will assume responsibility for the cluster by being designated as the leader. If the leader goes offline, the other nodes will notice its absence and a new leader will be elected. You ideally need an odd number of hosts, such as 3, 5, or 7, to optimize the election process.<\/li>\n<li><strong>Using an external etcd datastore.<\/strong> \u2013 This approach is similar to the stacked model but with one key difference. It relies on an external etcd instance which will be shared by your control plane nodes. This can avoid wasted data replication. You should consider manually setting up replication of the etcd cluster so it doesn\u2019t become a separate point of failure.<\/li>\n<\/ol>\n<p>Kubernetes now <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/setup\/production-environment\/tools\/kubeadm\/high-availability\/#steps-for-the-first-control-plane-node\">has good support<\/a> for clusters with several control planes. If you administer your own cluster, you can add another control plane node by simply including the <code>--control-plane<\/code> flag when you run the <code>kubeadm join<\/code> command:<\/p>\n<pre>$ kubeadm join &lt;cluster-control-plane-leader-ip&gt;:6443 \\&#13;\n    --token &lt;cluster-join-token&gt;&#13;\n    --discovery-token-ca-cert-hash sha256:&lt;cluster-discovery-token-ca-cert-hash&gt; \\ &#13;\n    --certificate-key &lt;cluster-certificate-key&gt; \\&#13;\n    --control-plane<\/pre>\n<h2 id=\"summary\"><span class=\"ez-toc-section\" id=\"Summary\"><\/span>Summary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The Kubernetes control plane is responsible for maintaining cluster-level operations. It oversees your worker nodes, handles API requests, and applies actions inside the cluster to achieve your desired state.<\/p>\n<p>When the control plane goes down, these functions will be unavailable but you should be able to continue using existing Pods for a limited period. The control plane is what stitches nodes together to form a cluster; without it, nodes are forced to operate independently, without any awareness of each other.<\/p>\n<p>As the control plane\u2019s a centralized single point of failure, mission-critical clusters need to replicate it across multiple master nodes to maximize reliability. Multi-master clusters distribute cluster management functions in a similar way to how worker nodes make your containers highly available. Although they can be trickier to set up, the extra redundancy is worth the effort. A highly available control plane is also offered as a feature of many cloud providers\u2019 managed Kubernetes offerings.<\/p>\n<\/div>\n<p><script>\n setTimeout(function(){\n  !function(f,b,e,v,n,t,s)\n  {if(f.fbq)return;n=f.fbq=function(){n.callMethod?\n  n.callMethod.apply(n,arguments):n.queue.push(arguments)};\n  if(!f._fbq)f._fbq=n;n.push=n;n.loaded=!0;n.version='2.0';\n  n.queue=[];t=b.createElement(e);t.async=!0;\n  t.src=v;s=b.getElementsByTagName(e)[0];\n  s.parentNode.insertBefore(t,s) } (window, document,'script',\n  'https:\/\/connect.facebook.net\/en_US\/fbevents.js');\n   fbq('init', '335401813750447');\n   fbq('track', 'PageView');\n  },3000);\n<\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/www.howtogeek.com\/devops\/what-happens-during-a-kubernetes-control-plane-failure\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;What Happens During a Kubernetes Control Plane Failure?&#8221; Kubernetes is the leading orchestrator for distributing container instances across multiple physical nodes. The nodes are managed by the Kubernetes control plane, a collection of components which maintain the cluster\u2019s state, respond to changing conditions, and handle scheduling decisions. It\u2019s essential to understand the control plane\u2019s role&#8230;<\/p>\n","protected":false},"author":1,"featured_media":492520,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/www.howtogeek.com\/wp-content\/uploads\/2022\/05\/Kubernetes-New.jpg?height=200p&trim=2,2,2,2","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-492519","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/492519","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=492519"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/492519\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/492520"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=492519"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=492519"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=492519"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}