{"id":292785,"date":"2021-07-07T13:00:21","date_gmt":"2021-07-07T10:00:21","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/how-to-create-a-semaphore-in-bash-cloudsavvy-it\/"},"modified":"2021-07-07T13:00:21","modified_gmt":"2021-07-07T10:00:21","slug":"how-to-create-a-semaphore-in-bash-cloudsavvy-it","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/how-to-create-a-semaphore-in-bash-cloudsavvy-it\/","title":{"rendered":"#How to Create a Semaphore in Bash \u2013 CloudSavvy IT"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a4b828622b0c\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a4b828622b0c\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-a-semaphore-in-bash-cloudsavvy-it\/#What_Is_a_Semaphore\" >What Is a Semaphore?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-a-semaphore-in-bash-cloudsavvy-it\/#Implementing_a_Semaphore_in_Bash_Easy_or_Not\" >Implementing a Semaphore in Bash: Easy, or Not?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-a-semaphore-in-bash-cloudsavvy-it\/#Creating_a_Bash_Semaphore\" >Creating a Bash Semaphore<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-create-a-semaphore-in-bash-cloudsavvy-it\/#Wrapping_up\" >Wrapping up<\/a><\/li><\/ul><\/nav><\/div>\n<p><strong>&#8220;#How to Create a Semaphore in Bash \u2013 CloudSavvy IT&#8221;<\/strong><\/p>\n<div id=\"article-content-area\">\n<img loading=\"lazy\" decoding=\"async\" class=\"type:primaryImage alignnone size-full wp-image-4038\" src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2017\/07\/add8ac45.png?width=1198&amp;trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Bash Shell\" width=\"1400\" height=\"600\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Are you developing a multi-threaded <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lication? Sooner or later, you will likely need to use a Semaphore. In this article, you will learn what a Semaphore is, how to create\/implement one in Bash, and more.<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"What_Is_a_Semaphore\"><\/span>What Is a <i>Semaphore<\/i>?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>A semaphore is a programming construct that is used in computer programs that employ multiple processing threads (computer processing threads which each execute source code from the same program or same suite of programs) to achieve exclusive use of a common resource at a given point in time. Said in a much easier way, think about this like \u201cone at the time, please.\u201d<\/p>\n<p>A semaphore was first defined towards the end of the 1960s by the late computer scientist Edsger Dijkstra from Rotterdam in the Netherlands. You have likely used semaphores many times in your life without specifically realizing you were doing so!<\/p>\n<p>Staying in the Netherlands for a bit, a country full of small waterways and many movable bridges (called a <i>drawbridge<\/i> in American English), one can see many excellent real-life world examples of a semaphore; consider the handle of a drawbridge operator: up or down. That handle is the semaphore variable protecting either the waterway or the road from accidents. Thus, the waterway and roads could be seen as other variables protected by the semaphore.<\/p>\n<p>If the handle is up and the bridge is open, the exclusive use of the water\/road intersection is given to the ship or ships passing through the channel of water. When the handle is down, the bridge is closed, and the exclusive use of the water\/road intersection is given to the cars which pass over the bridge.<\/p>\n<p>The semaphore variable can control access to another set of variables. For example, the up state of the handle prevents the <code>$cars_per_minute<\/code> and <code>$car_toll_booth_total_income<\/code> variables from being updated, etc.<\/p>\n<p>We can take the example a little further and clarify that when the handle operates up or down, a matching light is visible to both ship captains on the water and people driving in cars and trucks on the road: all operators can read a common variable for a specific state.<\/p>\n<p>While the scenario described here is not only a semaphore, but it is also a simple mutex. A mutex is another common programming construct that is very similar to a semaphore, with the additional condition that a mutex may only be unlocked by the same task or thread which locked it. Mutex stands for \u201cmutually exclusive.\u201d<\/p>\n<p>In this case, this applies to our example as the bridge operator is the only one with control over our semaphore and mutex up\/down handle. In contrast, if the guardhouse at the end of the bridge had a bridge override switch, we would still have a semaphore setup, but not a mutex.<\/p>\n<p>Both constructs are used regularly in computer programming when multiple threads are used to ensure only a single process or task access a given resource at any none time. Some semaphores may be ultra-fast switching, for example, when employed in multithreaded financial market trading software, and some may be much slower, only changing state every few minutes, like when used in an automated drawbridge or in a road train crossing.<\/p>\n<p>Now that we have a better understanding of semaphores let\u2019s implement one in Bash.<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Implementing_a_Semaphore_in_Bash_Easy_or_Not\"><\/span>Implementing a Semaphore in Bash: Easy, or Not?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Implementing a semaphore in Bash is so easy that it can even be done directly from the command line, or so it seems\u2026<\/p>\n<p>Let\u2019s start simple.<\/p>\n<pre>BRIDGE=up&#13;\nif [ \"${BRIDGE}\" = \"down\" ]; then echo \"Cars may pass!\"; else echo \"Ships may pass!\"; fi&#13;\nBRIDGE=down&#13;\nif [ \"${BRIDGE}\" = \"down\" ]; then echo \"Cars may pass!\"; else echo \"Ships may pass!\"; fi&#13;\n<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-11910\" src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/06\/5004a2bb.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Command line example of bridge status update and output\" width=\"785\" height=\"120\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>In this code, the variable <code>BRIDGE<\/code> holds our bridge status. When we set it to <code>up<\/code>, ships may pass and when we set it to <code>down<\/code>, cars may pass. We could also read out the value of our variable at any point to see whether the bridge is really up or down. The shared\/common resource, in this case, is our bridge.<\/p>\n<p>However, this example is single-threaded, and thus we never ran into a s<i>emaphore required<\/i> situation. Another way to think about this is that our variable can never be <code>up<\/code> and <code>down<\/code> at the exact same point in time as code is executed sequentially, i.e., step by step.<\/p>\n<p>Another thing to note is that we did not really control access to another variable (like a semaphore usually would do), and so our <code>BRIDGE<\/code> variable is not really a true semaphore variable, though it comes close.<\/p>\n<p>Finally, as soon as we introduce multiple threads which can affect the <code>BRIDGE<\/code> variable we run into issues. For example, what if, directly after the <code>BRIDGE=up<\/code> command, another thread issues <code>BRIDGE=down<\/code> which would then result in the message <code>Cars may pass!<\/code> output, even though the first thread would expect the bridge to be <code>up<\/code>, and in reality the bridge is still moving. Dangerous!<\/p>\n<p>You can see how things can quickly become murky and confusing, not to mention complex, when working with multiple threads.<\/p>\n<p>The situation where multiple threads try and update the same variable either at the same time or at least close enough in time for another thread to get the situation wrong (which in the case of drawbridges can be quite a while) is called a <i>race condition<\/i>: two threads racing to updating or reporting some variable or status, with the result that one or more threads may get it quite wrong.<\/p>\n<p>We can make this code much better by flowing the code into procedures and by using a real semaphore variable which will restrict access to our <code>BRIDGE<\/code> variable depending on the situation.<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Creating_a_Bash_Semaphore\"><\/span>Creating a Bash Semaphore<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Implementing a full multi-threaded, which is thread-safe (a computing term to describe software that is thread-safe or developed in such a way that threads cannot negatively\/incorrectly affect each other when they should not) is not an easy feat. Even a well-written program that employs semaphores is not guaranteed to be fully thread-safe.<\/p>\n<p>The more threads there are, and the higher the frequency and complexity of thread interactions, the more likely it is that there will be race conditions.<\/p>\n<p>For our small example, we will look at defining a Bash semaphore when one of the drawbridge operators lowers a bridge handle, thereby indicating he or she wants to lower the bridge. Avid readers may have noticed the reference to <i>operator<strong>s<\/strong><\/i> instead of <i>operator<\/i>: there are now multiple operators which can lower the bridge. In other words, there are multiple threads or tasks which all run at the same time.<\/p>\n<pre>#!\/bin\/bash&#13;\n&#13;\nBRIDGE_SEMAPHORE=0&#13;\n&#13;\nlower_bridge(){  # An operator put one of the bridge operation handles downward (as a new state).&#13;\n  # Assume it was previously agreed between operators that as soon as one of the operators &#13;\n  # moves a bridge operation handle that their command has to be executed, either sooner or later&#13;\n  # hence, we commence a loop which will wait for the bridge to become available for movement&#13;\n  while true; do&#13;\n    if [ \"${BRIDGE_SEMAPHORE}\" -eq 1 ]; then&#13;\n      echo \"Bridge semaphore locked, bridge moving or other issue. Waiting 2 minutes before re-check.\"&#13;\n      sleep 120&#13;\n      continue  # Continue loop&#13;\n    elif [ \"${BRIDGE_SEMAPHORE}\" -eq 0 ]; then   &#13;\n      echo \"Lower bridge command accepted, locking semaphore and lowering the bridge.\"&#13;\n      BRIDGE_SEMAPHORE=1&#13;\n      execute_lower_bridge&#13;\n      wait_for_bridge_to_come_down&#13;\n      BRIDGE='down'&#13;\n      echo \"Bridge lowered, ensuring at least 5 minutes pass before next allowed bridge movement.\"&#13;\n      sleep 300&#13;\n      echo \"5 Minutes passed, unlocking semaphore (releasing bridge control)\"&#13;\n      BRIDGE_SEMAPHORE=0&#13;\n      break  # Exit loop&#13;\n    fi&#13;\n  done&#13;\n}&#13;\n<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-11911\" src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/06\/19ee203f.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"A semaphore implementation in Bash\" width=\"870\" height=\"484\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Here we have a <code>lower_bridge<\/code> function which will do a number of things. Firstly, let\u2019s assume another operator has recently moved the bridge up within the last minute. As such, there is another thread executing code in a function similar to this one called <code>raise_bridge<\/code>.<\/p>\n<p>In fact, that function has finished raising the bridge but has instituted a mandatory 5-minute wait which all operators previously agreed on and which was hard coded into the source code: it prevents the bridge from going up\/down all the time. You can also see this mandatory 5-minute wait implemented in this function as <code>sleep 300<\/code>.<\/p>\n<p>So, when that <code>raise_bridge<\/code> function is operating, it will have set the semaphore variable <code>BRIDGE_SEMAPHORE<\/code> to 1, just like we do in the code here (directly after the <code>echo \"Lower bridge command accepted, locking semaphore and lowering bridge\"<\/code> command), and \u2013 by means of the first <code>if<\/code> conditional check in this code \u2013 the infinite loop present in this function will continue (ref <code>continue<\/code> in the code) to loop, with pauses of 2 minutes, as the <code>BRIDGE_SEMAPHORE<\/code> variable is 1.<\/p>\n<p>As soon as that <code>raise_bridge<\/code> function finishes raising the bridge and finishing its five minute sleep, it will set the <code>BRIDGE_SEMAPHORE<\/code> to 0, allowing our <code>lower_bridge<\/code> function co commence executing the functions <code>execute_lower_bridge<\/code> and subsequent <code>wait_for_bridge_to_come_down<\/code> whilst having first re-locked our semaphore to <code>1<\/code> to prevent other functions from taking over bridge control.<\/p>\n<p>There are, however, shortcomings in this code, and race conditions that may have far-reaching consequences for bridge operators are possible. Can you spot any?<\/p>\n<p>The <code>\"Lower bridge command accepted, locking semaphore and lowering bridge\"<\/code> is not thread-safe!<\/p>\n<p>If another thread, for example <code>raise_bridge<\/code> is executing at the same time and trying to access the <code>BRIDGE_SEMAPHORE<\/code> variable, it could be (when <code>BRIDGE_SEMAPHORE=0<\/code> and both threads running reach their respective <code>echo<\/code>\u2018s at the exact same time that the bridge operators see <i>\u201cLower bridge command accepted, locking semaphore and lowering bridge\u201d<\/i> and <i>\u201cRaise bridge command accepted, locking semaphore and raising bridge\u201d<\/i>. Directly after each other on the screen! Scary, no?<\/p>\n<p>More scary still is the fact that both threads can proceed to <code>BRIDGE_SEMAPHORE=1<\/code>, and both threads can continue executing! (There is nothing to stop them from doing so) The reason is that there is not much protection yet for such scenarios. While this code implements a semaphore, it is thus by no means thread-safe. As stated, multi-threaded coding is complex and requires much expertise.<\/p>\n<p>While the timing required in this case is minimal (1-2 lines of code take only a few milliseconds to run), and given the likely low number of bridge operators, the possibility of this happening is very small. However, the fact it is possible is what makes it dangerous. Creating thread-safe code in Bash is not an easy feat.<\/p>\n<p>This could be improved further by, for example, introducing a pre-lock and\/or by introducing some form of delay with subsequent re-check (though this will likely require an additional variable) or making a regular re-check before actual bridge execution, etc. Another option is to make a priority queue or a counter variable that checks how many threads have locked control of the bridge etc.<\/p>\n<p>Another commonly used approach, for example, when running multiple bash scripts that could interact, is to use <code>mkdir<\/code> or <code>flock<\/code> as base locking operations. There are various examples of how to implement these available online, for example,\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/unix.stackexchange.com\/questions\/70\/what-unix-commands-can-be-used-as-a-semaphore-lock\">What Unix commands can be used as a semaphore\/lock?<\/a>.<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Wrapping_up\"><\/span>Wrapping up<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In this article, we have a look at what a semaphore is. We also briefly touched on the subject of a mutex. Finally, we looked at implementing a semaphore in Bash using the practical example of multiple bridge operators operating a movable bridge\/drawbridge. We also explored how complex implementing a reliable semaphore-based solution is.<\/p>\n<p>If you enjoyed reading this article, have a look at our Asserts, Errors, and Crashes: What\u2019s the Difference? article.\n<\/p><\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/www.cloudsavvyit.com\/11909\/how-to-create-a-semaphore-in-bash\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#How to Create a Semaphore in Bash \u2013 CloudSavvy IT&#8221; Are you developing a multi-threaded application? Sooner or later, you will likely need to use a Semaphore. In this article, you will learn what a Semaphore is, how to create\/implement one in Bash, and more. What Is a Semaphore? A semaphore is a programming construct&#8230;<\/p>\n","protected":false},"author":1,"featured_media":292786,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2017\/07\/add8ac45.png","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-292785","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/292785","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=292785"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/292785\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/292786"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=292785"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=292785"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=292785"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}