{"id":176750,"date":"2021-02-12T16:00:30","date_gmt":"2021-02-12T13:00:30","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/"},"modified":"2021-02-12T16:00:30","modified_gmt":"2021-02-12T13:00:30","slug":"how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/","title":{"rendered":"#How to Garbage Collect the GitLab Container Registry to Free Up Storage \u2013 CloudSavvy IT"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a413e3eb2ef6\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a413e3eb2ef6\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/#Setting_a_Cleanup_Policy\" >Setting a Cleanup Policy<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/#Using_the_API\" >Using the API<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/#Effects_of_the_Cleanup_Policy\" >Effects of the Cleanup Policy<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/#Garbage_Collection\" >Garbage Collection<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/#Removing_Untagged_Manifests_and_Layers\" >Removing Untagged Manifests and Layers<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/#Running_Garbage_Collection_on_a_Schedule\" >Running Garbage Collection on a Schedule<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage-cloudsavvy-it\/#Limitations_of_Garbage_Collection\" >Limitations of Garbage Collection<\/a><\/li><\/ul><\/nav><\/div>\n<p><strong>&#8220;#How to Garbage Collect the GitLab Container Registry to Free Up Storage \u2013 CloudSavvy IT&#8221;<\/strong><\/p>\n<div id=\"article-content-area\">\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-9546\" src=\"https:\/\/www.cloudsavvyit.com\/thumbcache\/0\/0\/9f62c3cc40c0ec502b5e7a70f86bf493\/p\/uploads\/2021\/02\/266bb4cf.jpg\" alt=\"Graphic showing the GitLab logo, a stylised fox head\" width=\"1602\" height=\"902\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>GitLab\u2019s Container Registry provides a convenient place to store your Docker images. Over time, the Container Registry can eat up your disk space as more layers are added. Here\u2019s how to free up storage by removing redundant material.<\/p>\n<p>The Container Registry lets you store Docker images alongside your project\u2019s source code. If you\u2019re keeping large images in your registry, you can find your storage cost quickly balloons beyond your expectations. GitLab retains each layer indefinitely, even after it becomes redundant.<\/p>\n<h2 id=\"setting-a-cleanup-policy\"><span class=\"ez-toc-section\" id=\"Setting_a_Cleanup_Policy\"><\/span>Setting a Cleanup Policy<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The first step in regaining your storage space is to configure a Container Registry Cleanup Policy. Cleanup Policies are <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lied individually to each project. This means you can customise the retention period to suit each codebase.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-9541\" src=\"https:\/\/www.cloudsavvyit.com\/thumbcache\/0\/0\/e1ab81082b7b35264e70453f105e7c7c\/p\/uploads\/2021\/02\/dddba120.jpg\" alt=\"\" width=\"371\" height=\"351\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Visit your project in GitLab and click the \u201cSettings\u201d link in the sidebar. Switch to the \u201cCI \/ CD\u201d category and expand the \u201cClean up image tags\u201d section near the bottom of the page.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-9542\" src=\"https:\/\/www.cloudsavvyit.com\/thumbcache\/0\/0\/1eee863ce7806871e55f23a1b83fe635\/p\/uploads\/2021\/02\/e91cc9cd.jpg\" alt=\"\" width=\"996\" height=\"252\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Toggle the \u201cEnabled\u201d button to the on position to activate the Cleanup Policy. Next, choose when to run the policy \u2013 \u201cevery day\u201d is a good default.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-9543\" src=\"https:\/\/www.cloudsavvyit.com\/thumbcache\/0\/0\/e204c72b52bd4811be5bb6f42c21077a\/p\/uploads\/2021\/02\/51b09904.jpg\" alt=\"\" width=\"874\" height=\"321\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>The next section, \u201cKeep these tags\u201d, allows you to define tags which the Cleanup Policy will leave alone. The two options, \u201ckeep the most recent\u201d and \u201ckeep tags matching\u201d, are independent of each other. You could choose to keep <code>dev<\/code> and <code>nightly<\/code>, supplemented by the five most recent tags. The <code>latest<\/code> tag is <em>always<\/em> included, in addition to any set tags.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-9544\" src=\"https:\/\/www.cloudsavvyit.com\/thumbcache\/0\/0\/cc9e74f1e372f74e6196f1f1b447229d\/p\/uploads\/2021\/02\/fb0c5395.jpg\" alt=\"\" width=\"875\" height=\"323\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>The following section, \u201cRemove these tags\u201d, defines the whitelist of tags to remove. Tags which don\u2019t match the regex pattern won\u2019t be touched. Adjust the \u201cRemove tags older than\u201d value to set the maximum lifetime of each tag, before it gets cleaned up. When you\u2019re done, click the green \u201cSave\u201d button.<\/p>\n<h2 id=\"using-the-api\"><span class=\"ez-toc-section\" id=\"Using_the_API\"><\/span>Using the API<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Applying cleanup policies through the web UI can quickly get tedious. Use the API instead if you\u2019re changing multiple projects.<\/p>\n<pre>curl --request PUT --header 'Content-Type: application\/json;charset=UTF-8' --header \"PRIVATE-TOKEN: &lt;access_token&gt;\" --data-binary '{\"container_expiration_policy_attributes\":{\"cadence\":\"1month\",\"enabled\":true,\"keep_n\":1,\"older_than\":\"14d\",\"name_regex\":\"\",\"name_regex_delete\":\".*\",\"name_regex_keep\":\"latest' \"https:\/\/gitlab.example.com\/api\/v4\/projects\/&lt;project_id&gt;\"<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-9545\" src=\"https:\/\/www.cloudsavvyit.com\/thumbcache\/0\/0\/3fbef67ace1549cc77a5db79ce48581e\/p\/uploads\/2021\/02\/91d5da6e.jpg\" alt=\"\" width=\"422\" height=\"201\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>You\u2019ll need to <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/docs.gitlab.com\/ee\/user\/profile\/personal_access_tokens.html\">generate an API access token<\/a> by heading to your Profile page in GitLab. Use the token as <code>&lt;access_token&gt;<\/code> in the command above. Adjust the URL to point to your project \u2013 its ID can be found on its project page within GitLab.<\/p>\n<p>Running the command above will apply a registry cleanup policy that runs every month and cleans images older than 14 days. The <code>latest<\/code> tag, and the most recent tag (<code>keep_n<\/code>), will be retained; all others will be eligible for removal (<code>.*<\/code>).<\/p>\n<h2 id=\"effects-of-the-cleanup-policy\"><span class=\"ez-toc-section\" id=\"Effects_of_the_Cleanup_Policy\"><\/span>Effects of the Cleanup Policy<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The Cleanup Policy handles untagging of images based on the criteria you set. The tags will be deleted from your container registry. They\u2019ll no longer show up in your project\u2019s Container Registry screen and won\u2019t be pullable by Docker clients.<\/p>\n<p>Untagging an image isn\u2019t the same as actually deleting it though. The Cleanup Policy <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/docs.gitlab.com\/ee\/user\/packages\/container_registry\/#cleanup-policy\">doesn\u2019t recycle data<\/a>, so you may still see high storage use even after pruning disused tags.<\/p>\n<p>This is because the image layers remain on your GitLab server, cached for future reference. To remove the data for good, you should next run the Container Registry Garbage Collection procedure.<\/p>\n<h2 id=\"garbage-collection\"><span class=\"ez-toc-section\" id=\"Garbage_Collection\"><\/span>Garbage Collection<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Running Garbage Collection will delete all image layers which aren\u2019t linked to a tag. This will result in the removal of images that got untagged by your cleanup policy. It can also dispose of old layers which became redundant when you pushed a new version of a tag.<\/p>\n<p>Garbage Collection must be invoked manually via GitLab\u2019s command line interface. Connect to your GitLab server over SSH and run the following command:<\/p>\n<pre>sudo gitlab-ctl registry-garbage-collect<\/pre>\n<p>The Garbage Collection process will run. Any unused tags within your Container Registry will be recycled. Garbage Collection looks for untagged images across your entire GitLab instance.<\/p>\n<p>Assuming you let the cleanup policy run first, you should now see a healthy reduction in storage use. If it\u2019s your first time running garbage collection on a frequently used GitLab installation, you might have reclaimed several gigabytes of space.<\/p>\n<h2 id=\"removing-untagged-manifests-and-layers\"><span class=\"ez-toc-section\" id=\"Removing_Untagged_Manifests_and_Layers\"><\/span>Removing Untagged Manifests and Layers<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>You can reclaim even more space by instructing garbage collection to also remove untagged image manifests and unreferenced layers. This is a more destructive operation, although it\u2019s normally what you\u2019re expecting to see.<\/p>\n<pre>sudo gitlab-ctl registry-garbage-collect -m<\/pre>\n<p>Adding the <code>-m<\/code> flag will delete any layer not directly associated with a tagged image manifest. This results in the loss of cached layers and inter<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">media<\/a>te build steps.<\/p>\n<p>By default, Docker and the GitLab Container Registry retain all created layers, even if they\u2019re no longer referenced. This means you can always retrieve a previously known layer using its unique <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/docs.docker.com\/engine\/reference\/commandline\/images\/\">content-addressable identifier<\/a>, even if no longer possesses a tag.<\/p>\n<p>This is why removing these layers is not enabled by default. You need to be aware of the implications before running the command as it could have serious consequences in some workflows. Nonetheless, using the <code>-m<\/code> switch is often desirable \u2013 it will free up much more disk space and shouldn\u2019t have any side effects if you only reference images using tag names.<\/p>\n<h2 id=\"running-garbage-collection-on-a-schedule\"><span class=\"ez-toc-section\" id=\"Running_Garbage_Collection_on_a_Schedule\"><\/span>Running Garbage Collection on a Schedule<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Cleanup Policies run automatically on the frequency you\u2019ve configured. Garbage Collection isn\u2019t setup by default, which is why a first run can deliver a dramatic reduction in storage utilisation.<\/p>\n<p>To run garbage collection on a schedule, you\u2019ll need to add the command to your system\u2019s <code>crontab<\/code>. Create a file <code>\/etc\/cron.d\/registry-garbage-collection<\/code> with the following content to run garbage collection every Monday at 2am:<\/p>\n<pre>SHELL=\/bin\/sh&#13;\nPATH=\/usr\/local\/sbin:\/usr\/local\/bin:\/sbin:\/bin:\/usr\/sbin:\/usr\/bin&#13;\n&#13;\n0 2 * * 1  root gitlab-ctl registry-garbage-collect<\/pre>\n<h2 id=\"limitations-of-garbage-collection\"><span class=\"ez-toc-section\" id=\"Limitations_of_Garbage_Collection\"><\/span>Limitations of Garbage Collection<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The time taken to perform garbage collection will depend on the amount of data there is to delete. Garbage Collection requires the Container Registry service to be stopped while it\u2019s running. This means your users won\u2019t be able to pull or push images until the process completes.<\/p>\n<p>You can reduce the impact of downtime by switching the registry into read-only mode, running the command and then switching back to read-write. The registry can remain running throughout but users won\u2019t be able to push images. In addition, switching modes requires GitLab to be \u201creconfigured\u201d (<code>sudo gitlab-ctl reconfigure<\/code>), which can in itself cause downtime depending on how your installation is setup.<\/p>\n<p>You need to edit the following lines in <code>\/etc\/gitlab\/gitlab.rb<\/code>:<\/p>\n<pre escape=\"true\">registry['storage'] = {&#13;\n    'maintenance' =&gt; {&#13;\n      'readonly' =&gt; {&#13;\n        'enabled' =&gt; true&#13;\n      }&#13;\n    }&#13;\n  }<\/pre>\n<p>Run <code>sudo gitlab-ctl reconfigure<\/code>, then use one of the garbage collection commands. Once it\u2019s done, disable read-only mode by changing the <code>enabled<\/code> line in your <code>gitlab.rb<\/code> back to <code>false<\/code>, then reconfigure GitLab again.\n<\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/www.cloudsavvyit.com\/9540\/how-to-garbage-collect-the-gitlab-container-registry-to-free-up-storage\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#How to Garbage Collect the GitLab Container Registry to Free Up Storage \u2013 CloudSavvy IT&#8221; GitLab\u2019s Container Registry provides a convenient place to store your Docker images. Over time, the Container Registry can eat up your disk space as more layers are added. Here\u2019s how to free up storage by removing redundant material. The Container&#8230;<\/p>\n","protected":false},"author":1,"featured_media":176751,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/www.cloudsavvyit.com\/thumbcache\/0\/0\/9f62c3cc40c0ec502b5e7a70f86bf493\/p\/uploads\/2021\/02\/266bb4cf.jpg","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-176750","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/176750","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=176750"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/176750\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/176751"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=176750"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=176750"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=176750"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}