{"id":371863,"date":"2021-11-24T15:45:55","date_gmt":"2021-11-24T12:45:55","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/should-you-use-an-s3-alternative-for-object-storage-cloudsavvy-it\/"},"modified":"2021-11-24T15:45:55","modified_gmt":"2021-11-24T12:45:55","slug":"should-you-use-an-s3-alternative-for-object-storage-cloudsavvy-it","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/should-you-use-an-s3-alternative-for-object-storage-cloudsavvy-it\/","title":{"rendered":"#Should You Use an S3 Alternative For Object Storage? \u2013 CloudSavvy IT"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a2f38c9b05b5\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a2f38c9b05b5\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/should-you-use-an-s3-alternative-for-object-storage-cloudsavvy-it\/#What_Is_S3_Compatible\" >What Is S3 Compatible?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/should-you-use-an-s3-alternative-for-object-storage-cloudsavvy-it\/#Digital_Ocean\" >Digital Ocean<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/should-you-use-an-s3-alternative-for-object-storage-cloudsavvy-it\/#Self_Host\" >Self Host<\/a><\/li><\/ul><\/nav><\/div>\n<p><strong>&#8220;#Should You Use an S3 Alternative For Object Storage? \u2013 CloudSavvy IT&#8221;<\/strong><\/p>\n<div id=\"article-content-area\">\n<figure style=\"width: 1200px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"type:primaryImage wp-image-14848 size-full\" srcset=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/11\/c387c310.png?width=398&amp;trim=1,1&amp;bg-color=000&amp;pad=1,1 400w, https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/11\/c387c310.png?width=1198&amp;trim=1,1&amp;bg-color=000&amp;pad=1,1 1200w\" sizes=\"auto, 400w, 1200w\" src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/11\/c387c310.png?width=1198&amp;trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"\" width=\"1200\" height=\"675\" data-crediturl=\"https:\/\/www.shutterstock.com\/image-photo\/data-center-technician-running-maintenance-programme-1394052998\" data-credittext=\"Shutterstock\/Gorodenkoff\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><figcaption class=\"wp-caption-text\"><span class=\"type:primaryImage imagecredit\"><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.shutterstock.com\/image-photo\/data-center-technician-running-maintenance-programme-1394052998\">Shutterstock\/Gorodenkoff<\/a><\/span><\/figcaption><\/figure>\n<p>Amazon\u2019s Simple Storage Service (S3) provides a very useful interface for storing objects in redudant cloud storage, where you don\u2019t have to worry about the underlying hardware. On top of being a service offered by Amazon, it\u2019s also an industry standard API, and there are many services compatible with it.<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"What_Is_S3_Compatible\"><\/span>What Is S3 Compatible?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In many cases, if you move to another cloud provider, you will have to rework a lot of your <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lication. But, if you\u2019re using S3 as your object storage backend, you\u2019ll be able to move seamlessly to many other services.<\/p>\n<p>This is because S3 is an open API standard. AWS\u2019s Simple Storage Service is just an implementation of this standard; it\u2019s native, and obviously will have the best support, but there are other services that will offer acceptable performance and stability, often for lower cost.<\/p>\n<p>Switching to these services is easy\u2014you simply have to change the URL endpoint your application uses, and it\u2019s usually good to go after some minor tweaks to key handling. You will have to <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.digitalocean.com\/community\/tutorials\/how-to-migrate-from-amazon-s3-to-digitalocean-spaces-with-rclone?tag=reviewgeek-20\">migrate your data with rclone<\/a>, but it\u2019s not a hard process, just a long one in some cases.<\/p>\n<p>It\u2019s no secret AWS is expensive. S3 is no different, and while\u00a0<em>storing<\/em> files is very cheap, actually accessing those files is not. In a typical read\/write heavy workload serving live files to users, storing the files is usually cheap; the highest costs are actually AWS data transfer charges, and S3 request charges:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-14849\" src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/11\/07304508.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"\" width=\"1167\" height=\"432\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Seeing a Cost Explorer breakdown like this, you may be tempted to consider a third party service that will be cheaper on the data transfer charges for your workload.<\/p>\n<p>The two major competitors to AWS S3 are from Google and Microsoft. Google has their uncreatively named \u201c<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/cloud.google.com\/storage\">Cloud Storage<\/a>,\u201d and Microsoft Azure has <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/click.linksynergy.com\/deeplink?id=2QzUaswX1as&amp;mid=24542&amp;u1=csit\/14847&amp;murl=https%3A%2F%2Fazure.microsoft.com%2Fen-us%2Fservices%2Fstorage%2Fblobs%2F\">Azure Blob Storage<\/a>. Google\u2019s storage is S3 compatible, <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/cloud.google.com\/storage\/docs\/migrating\">and is relatively easy to migrate too.<\/a>\u00a0Azure, on the other hand, is not S3 compatible, though there are<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/github.com\/gaul\/s3proxy\"> tools like S3Proxy that can patch them together<\/a>.<\/p>\n<p>However, all of the storage services from the big three cloud providers will charge you high fees for data. They\u2019re designed for enterprise customers, and if you\u2019re a small business trying to minimize your costs, you should look elsewhere. There are other alternative cloud providers like <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.digitalocean.com\/\">Digital Ocean<\/a> and <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.vultr.com\/\">Vultr<\/a> that offer more streamlined pricing models with similar quality service.<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Digital_Ocean\"><\/span>Digital Ocean<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-14851\" src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/11\/5d0b0262.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"\" width=\"839\" height=\"318\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Digital Ocean is a cloud provider designed to be simple. While it doesn\u2019t offer as many features as major providers like AWS, it usually does right by the services it does offer. One of these services is object storage, with buckets being called <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/docs.digitalocean.com\/products\/spaces\/\">Spaces<\/a>, and it\u2019s what we will recommend if you\u2019re looking to move away from AWS.<\/p>\n<p>Spaces are pretty simple. The base rate is $5 a month, and includes\u00a0250 GB of storage along with a whole TB of outbound data transfer. This is an insanely good deal\u2014the same usage would cost over $90 on AWS S3.<\/p>\n<p>Additional data storage is $0.02 per GB, fairly standard compared to S3 (although higher if you plan to use cheaper archive storage), and additional data is priced reasonable at\u00a0$0.01 per GB transfered, which is 90% cheaper than AWS pricing.<\/p>\n<p>Of course, this comes with a few limits, and unfortunately there are a lot more downsides and strings attached to this great deal.<\/p>\n<ul>\n<li>750 requests, per IP address, to all of your Spaces.<\/li>\n<li>150 combined operations per second to any Space, not including GET requests.<\/li>\n<li>240 total operations including GET requests.<\/li>\n<li>5 PUT or COPY requests per 5 minutes to any individual object in a Space<\/li>\n<\/ul>\n<p>While these rate limits aren\u2019t great to have, the limits are fairly generous, and you\u2019re likely not going to hit them. If you are close to going over, you can minimize the effect of them by having multiple Spaces. If you\u2019re unsure, you can enable bucket metrics in S3 to check your current usage.<\/p>\n<p><strong>RELATED:<\/strong> <strong><em>How To Enable And View Request Metrics for an AWS S3 Bucket in CloudWatch<\/em><\/strong><\/p>\n<p>Also, Spaces with over 3 million objects, or 1.5 million with versioning enabled, may require \u201cintermittent maintenance periods\u201d to ensure consistent performance. However, I personally have a bucket with over 2 million versioned objects that has not appeared to have experienced any significant downtime over 6 months, so this may not be a common occurence.<\/p>\n<p>One major drawback of Spaces compared to S3 is the interface. Spaces is simple, and if you\u2019re looking to just upload your website content or store some basic files, the web interface will allow uploads, downloads, and editing of most settings. However, if you\u2019re storing lots of files or need advanced configuration, it\u2019s quite frankly pretty bad, and you\u2019ll have to mainly work with it over the S3 API.<\/p>\n<p>For example\u2014Spaces doesn\u2019t even have a web editor to select your Lifecycle configuration, which handles storing old versions of objects used as backups in case of user deletion. That also means there\u2019s no way of accessing or deleting versioned objects without listing the versions through the API and accessing them directly by version ID.<\/p>\n<p>They also don\u2019t have much documentation. To turn on versioning, for example, we had to consult S3\u2019s own documentation to use the mostly ignored\u00a0PutBucketVersioning endpoint, which thankfully is supported on Spaces despite it being ignored in DO\u2019s docs. You\u2019ll need to enable it through this endpoint:<\/p>\n<pre>PUT\u00a0?versioning&#13;\n&#13;\n&lt;VersioningConfiguration xmlns=\"http:\/\/s3.amazonaws.com\/doc\/2006-03-01\/\"&gt; &#13;\n  &lt;Status&gt;Enabled&lt;\/Status&gt; &#13;\n&lt;\/VersioningConfiguration&gt;<\/pre>\n<p>And then enable version expiration:<\/p>\n<pre>PUT ?lifecycle&#13;\n&#13;\n&lt;LifecycleConfiguration xmlns=\"http:\/\/s3.amazonaws.com\/doc\/2006-03-01\/\"&gt;&#13;\n  &lt;Rule&gt;&#13;\n    &lt;ID&gt;Bucket&lt;\/ID&gt;&#13;\n    &lt;Prefix&gt;*&lt;\/Prefix&gt;&#13;\n    &lt;Status&gt;Enabled&lt;\/Status&gt;&#13;\n    &lt;NoncurrentVersionExpiration&gt;&#13;\n      &lt;NoncurrentDays&gt;90&lt;\/NoncurrentDays&gt;&#13;\n    &lt;\/NoncurrentVersionExpiration&gt;&#13;\n  &lt;\/Rule&gt;&#13;\n&lt;\/LifecycleConfiguration&gt;<\/pre>\n<p>API keys are also very basic. You will not have granular control over individual buckets, objects, or anything else that comes with AWS IAM. This can be an issue if you plan to give keys to third parties.<\/p>\n<hr\/>\n<p>Overall, the Digital Ocean experience is definitely nowhere close to how good AWS\u2019s S3 is. But, if you\u2019re fine with the limits, and don\u2019t mind using the API for certain tasks, it can certainly save you a ton of money on bandwidth costs.<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Self_Host\"><\/span>Self Host<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-14857\" src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/11\/490a96c2.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"\" width=\"930\" height=\"318\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Since S3 is an open standard, it\u2019s also something you can host yourself, which will be preferable for many people. There are lots of tools to do this, <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/min.io\/\">but one of the best is MinIO<\/a>, which runs on Kubernetes (K8s).<\/p>\n<p>Being on K8s means you can run it on every public cloud, including running it through serverless K8s services like AWS EKS. But, you\u2019d still be subject to bandwidth costs in this case.<\/p>\n<p>Where MinIO really shines is with dedicated servers, hybrid cloud solutions, and running on on-premises datacenters. If you\u2019re paying for a dedicated network connection to a server, you won\u2019t be nickle-and-dimed if you saturate that connection. This can make self-hosted storage very cheap if you\u2019re planning on serving a lot of data to end users.<\/p>\n<p>Also, running on your own hardware isn\u2019t subject to the same limits as services like S3. You can host MinIO on blazing fast servers and get better performance in read\/write heavy workloads (and you won\u2019t be charged for requests). Of course, you will be required to pay the hardware costs for this performance.<\/p>\n<p>Where it falls flat is on redudancy\u2014because S3 stores your data in so many different places, it\u2019s basically guaranteed to always work and never lose your data, barring a giant meteor. MinIO, on the other hand, can be hosted on a single server or <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/docs.min.io\/docs\/distributed-minio-quickstart-guide.html\">through a distributed deployment<\/a>. If you\u2019re hosting on a single server, you will be screwed if your instance goes down. It\u2019s the cheapest option, but we highly recommend multiple servers in a cluster or at least doing some kind of backup to S3.<\/p>\n<p>MinIO is free to host under the GNU AGPL license, but you won\u2019t get any support. Corporate licenses start at $1000\/month and provide 24\/7 support as well as a \u201cPanic Button,\u201d which will have their data recovery engineers ready to help you fix serious server failures.\n<\/p><\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/www.cloudsavvyit.com\/14847\/should-you-use-an-s3-alternative-for-object-storage\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#Should You Use an S3 Alternative For Object Storage? \u2013 CloudSavvy IT&#8221; Shutterstock\/Gorodenkoff Amazon\u2019s Simple Storage Service (S3) provides a very useful interface for storing objects in redudant cloud storage, where you don\u2019t have to worry about the underlying hardware. On top of being a service offered by Amazon, it\u2019s also an industry standard API,&#8230;<\/p>\n","protected":false},"author":1,"featured_media":371864,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/11\/c387c310.png","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-371863","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/371863","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=371863"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/371863\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/371864"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=371863"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=371863"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=371863"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}