{"id":485393,"date":"2022-08-17T11:04:36","date_gmt":"2022-08-17T08:04:36","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/understanding-and-configuring-kubernetes-garbage-collection\/"},"modified":"2022-08-17T11:04:36","modified_gmt":"2022-08-17T08:04:36","slug":"understanding-and-configuring-kubernetes-garbage-collection","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/","title":{"rendered":"#Understanding and Configuring Kubernetes Garbage Collection"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_89 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6af76c2f4aa42\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6af76c2f4aa42\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/#%E2%80%9CUnderstanding_and_Configuring_Kubernetes_Garbage_Collection%E2%80%9D\" >&#8220;Understanding and Configuring Kubernetes Garbage Collection&#8221;<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/#Basic_Garbage_Collection\" >Basic Garbage Collection<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/#Owner_References\" >Owner References<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/#Cascading_Deletion_Mechanisms\" >Cascading Deletion Mechanisms<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/#Disabling_Garbage_Collection_of_Dependents\" >Disabling Garbage Collection of Dependents<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/#Garbage_Collection_and_Finalizers\" >Garbage Collection and Finalizers<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/#Customizing_Garbage_Collection_of_Unused_Resources\" >Customizing Garbage Collection of Unused Resources<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/buradabiliyorum.com\/en\/understanding-and-configuring-kubernetes-garbage-collection\/#Summary\" >Summary<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"%E2%80%9CUnderstanding_and_Configuring_Kubernetes_Garbage_Collection%E2%80%9D\"><\/span>&#8220;Understanding and Configuring Kubernetes Garbage Collection&#8221;<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<div>\n<img loading=\"lazy\" decoding=\"async\" class=\"type:primaryImage alignnone size-full wp-image-803403\" data-pagespeed-no-defer=\"\" src=\"https:\/\/www.howtogeek.com\/wp-content\/uploads\/2022\/05\/Kubernetes.jpg?width=1198&amp;trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Kubernetes logo\" width=\"1602\" height=\"902\"\/><\/p>\n<p>Garbage collection is the process by which Kubernetes removes dependent objects after you delete a resource. It also handles automatic clean-up of redundant items in your cluster. Finished containers and disused images are periodically removed to prevent large numbers of unnecessary resources accumulating over time.<\/p>\n<p>Several garbage collection strategies can be deployed depending on the type of object that\u2019s being deleted. Garbage collection of unused resources is configurable by Kubelet settings on your cluster\u2019s worker nodes. In this article, we\u2019ll explain the different forms of garbage collection and how you can customize clean-up settings.<\/p>\n<h2 id=\"basic-garbage-collection\"><span class=\"ez-toc-section\" id=\"Basic_Garbage_Collection\"><\/span>Basic Garbage Collection<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Kubernetes garbage collection handles the removal of dependent objects, old containers, and unused images. Kubelet runs cleaning operations periodically while its service is active. It will delete stopped containers, such as those from finished Jobs, and remove unused images once a certain disk utilization threshold is reached.<\/p>\n<p>Garbage collection of cluster resources in response to object deletions uses a <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/architecture\/garbage-collection\/#cascading-deletion\">cascading mechanism<\/a>. This ensures an object\u2019s children are removed alongside it, preventing orphans from existing. You can control this process by requesting a specific cascade order when you initiate deletion.<\/p>\n<h2 id=\"owner-references\"><span class=\"ez-toc-section\" id=\"Owner_References\"><\/span>Owner References<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Parent-child relationships are expressed by Kubernetes <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/overview\/working-with-objects\/owners-dependents\">owner references<\/a>. Owners are responsible for the objects nested within them. This creates a dependency graph which informs the garbage collection process.<\/p>\n<p>Kubernetes uses owner reference data to establish which resources should be removed before a target is deleted. As an example, Pods that are part of a ReplicaSet will have an owner reference that defines this link. Deleting the ReplicaSet will automatically remove the Pods within it too.<\/p>\n<p>Owner references are set on objects in their <code>metadata.ownerReference<\/code> manifest field. You shouldn\u2019t normally need to inspect or manually change this field as owner references are automatically managed by Kubernetes controllers.<\/p>\n<p>Children can block their parent\u2019s deletion by setting the <code>metadata.blockOwnerDeletion<\/code> field to the boolean value <code>true<\/code>. This will prevent parents in the object\u2019s <code>ownerReference<\/code> chain from being deleted until the object has itself been removed.<\/p>\n<h2 id=\"cascading-deletion-mechanisms\"><span class=\"ez-toc-section\" id=\"Cascading_Deletion_Mechanisms\"><\/span>Cascading Deletion Mechanisms<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The owner reference chain means <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/tasks\/administer-cluster\/use-cascading-deletion\">there\u2019s two ways<\/a> in which object deletions can proceed:<\/p>\n<ul>\n<li><strong>Background deletion:<\/strong> The target object is deleted first, then its dependents.<\/li>\n<li><strong>Foreground deletion:<\/strong> The dependents are deleted first, then the target object.<\/li>\n<\/ul>\n<p>Consider a deletion command such as the following:<\/p>\n<pre>$ kubectl delete deployment\/demo-<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a><\/pre>\n<p>The background deletion method will im<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">media<\/a>tely delete the <code>demo-app<\/code> Deployment object. This will leave its Pods running in your cluster. They\u2019ll be cleaned up automatically in the background as part of the garbage collection process.<\/p>\n<p>Foreground deletion begins by marking the <code>demo-app<\/code> Deployment as a \u201cdeletion in progress.\u201d It then deletes all the dependent objects that exist within it. The <code>demo-app<\/code> Deployment will remain visible in your cluster until its Pods have been removed. The Deployment will be cleaned up once the dependents are gone.<\/p>\n<p>Background cascading is the default mechanism. It offers an immediate result that removes the target object from your cluster without delay. Foreground deletion can be more desirable in some scenarios though, such as when you want a Deployment or ReplicaSet to be deleted only after its Pods have been destroyed. This avoids the brief period where the Pods are effectively orphaned, with their owner ReplicaSet already gone.<\/p>\n<p>You can opt-in to foreground deletion by passing the <code>--cascade=foreground<\/code> flag to the <code>kubectl delete<\/code> command:<\/p>\n<pre>$ kubectl delete deployment\/demo-app --cascade=foreground<\/pre>\n<p>The Pods will be deleted, then the <code>demo-app<\/code> deployment.<\/p>\n<h2 id=\"disabling-garbage-collection-of-dependents\"><span class=\"ez-toc-section\" id=\"Disabling_Garbage_Collection_of_Dependents\"><\/span>Disabling Garbage Collection of Dependents<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>There is a third cascade option: ignore dependent garbage collection altogether. This will orphan any dependents of the object, leaving them in your cluster but with their owner reference removed.<\/p>\n<pre>$ kubectl delete deployment\/demo-app --cascade=orphan<\/pre>\n<p>This command immediately deletes the <code>demo-app<\/code> Deployment but leaves its Pods intact. They\u2019ll keep running in your cluster until deleted separately, as part of another command. Although this behavior is rarely desirable, it can be helpful if you decide Pods no longer need to be in a Deployment or ReplicaSet.<\/p>\n<h2 id=\"garbage-collection-and-finalizers\"><span class=\"ez-toc-section\" id=\"Garbage_Collection_and_Finalizers\"><\/span>Garbage Collection and Finalizers<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Finalizers also affect how garbage collection operates. They prevent deletions from occurring until certain conditions are met.<\/p>\n<p>Many apparently \u201cstuck\u201d deletions are caused by pending finalizers. A finalizer that\u2019s not marked itself as finished will prevent the object being removed from your cluster. Once the finalizer has cleared, the Kubernetes control plane will proceed with the deletion.<\/p>\n<p>Some finalizers are included with Kubernetes, such as the <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/storage\/persistent-volumes\"><code>pv-protection<\/code> safeguard<\/a> against deleting actively used volumes. Others may be added by third-party controllers you add to your cluster. Overriding a finalizer isn\u2019t recommended as it could create an incorrect state that leads to unexpected behavior.<\/p>\n<h2 id=\"customizing-garbage-collection-of-unused-resources\"><span class=\"ez-toc-section\" id=\"Customizing_Garbage_Collection_of_Unused_Resources\"><\/span>Customizing Garbage Collection of Unused Resources<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Garbage collection of redundant containers and images occurs automatically on your worker nodes. Five parameters are available to tune the deletion thresholds:<\/p>\n<ul>\n<li><strong><code>maximum-dead-containers<\/code><\/strong> \u2013 The maximum number of old containers that can continue to exist after garbage collection is run. The default of <code>-1<\/code> removes the limit.<\/li>\n<li><strong><code>maximum-dead-containers-per-container<\/code><\/strong> \u2013 Set the maximum number of retained old containers on a per-container basis. This controls how many previous versions of a container stay in the cluster, after a new replacement is started.<\/li>\n<li><strong><code>minimum-container-ttl-duration<\/code><\/strong> \u2013 Containers become eligible for garbage collection this many minutes after they stop. The default value of <code>0<\/code> disables the delay.<\/li>\n<li><strong><code>image-gc-high-threshold<\/code><\/strong> \u2013 When disk usage reaches this percentage Kubelet will try to cull unused images\u2026<\/li>\n<li><strong><code>image-gc-low-threshold<\/code><\/strong> \u2013 \u2026to bring the usage down to this level.<\/li>\n<\/ul>\n<p>These parameters are set as Kubelet launch flags. Flags are typically placed into <code>\/var\/lib\/kubelet\/kubeadm-flags.env<\/code>:<\/p>\n<pre>KUBELET_KUBEADM_ARGS=\"--image-gc-high-threshold=60 --image-gc-low-threshold=50 --minimum-container-ttl-duration=5\"<\/pre>\n<p>This configuration will permit automatic deletion of containers that have been redundant for at least five minutes. Each garbage collection pass will also try to prune old images once 60% disk utilization is reached, aiming to reduce consumption down to 50%.<\/p>\n<p>You should restart the Kubelet process after you make these changes:<\/p>\n<pre>$ sudo systemctl daemon-reload&#13;\n$ sudo systemctl restart kubelet<\/pre>\n<p>As the settings are part of Kubelet, you\u2019ll need to apply them to each of your worker nodes in your cluster if you want configuration to be consistent across them all.<\/p>\n<h2 id=\"summary\"><span class=\"ez-toc-section\" id=\"Summary\"><\/span>Summary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Understanding the different forms of Kubernetes garbage collection helps you decide on the correct strategy when deleting objects from your cluster. Background and foreground cascades affect the order in which dependent resources are cleaned up. Relationships between objects are identified by their owner reference chains. Kubernetes also cleans up unused containers and images in the background. The thresholds for this behavior can be customized using Kubelet settings.<\/p>\n<p>For best results you should stick to the mechanisms that Kubernetes provides. The project <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/architecture\/garbage-collection\/#containers-images\">warns against using<\/a> third-party garbage collection tools as these can create state inconsistencies that cause problems in your cluster.<\/p>\n<\/div>\n<p><script>\n setTimeout(function(){\n  !function(f,b,e,v,n,t,s)\n  {if(f.fbq)return;n=f.fbq=function(){n.callMethod?\n  n.callMethod.apply(n,arguments):n.queue.push(arguments)};\n  if(!f._fbq)f._fbq=n;n.push=n;n.loaded=!0;n.version='2.0';\n  n.queue=[];t=b.createElement(e);t.async=!0;\n  t.src=v;s=b.getElementsByTagName(e)[0];\n  s.parentNode.insertBefore(t,s) } (window, document,'script',\n  'https:\/\/connect.facebook.net\/en_US\/fbevents.js');\n   fbq('init', '335401813750447');\n   fbq('track', 'PageView');\n  },3000);\n<\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/www.howtogeek.com\/devops\/understanding-and-configuring-kubernetes-garbage-collection\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;Understanding and Configuring Kubernetes Garbage Collection&#8221; Garbage collection is the process by which Kubernetes removes dependent objects after you delete a resource. It also handles automatic clean-up of redundant items in your cluster. Finished containers and disused images are periodically removed to prevent large numbers of unnecessary resources accumulating over time. Several garbage collection strategies&#8230;<\/p>\n","protected":false},"author":1,"featured_media":485394,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/www.howtogeek.com\/wp-content\/uploads\/2022\/05\/Kubernetes.jpg?height=200p&trim=2,2,2,2","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-485393","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/485393","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=485393"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/485393\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/485394"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=485393"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=485393"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=485393"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}