{"id":291101,"date":"2021-07-05T13:00:20","date_gmt":"2021-07-05T10:00:20","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/how-to-time-a-linux-program-cloudsavvy-it\/"},"modified":"2021-07-05T13:00:20","modified_gmt":"2021-07-05T10:00:20","slug":"how-to-time-a-linux-program-cloudsavvy-it","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/how-to-time-a-linux-program-cloudsavvy-it\/","title":{"rendered":"#How to Time a Linux Program \u2013 CloudSavvy IT"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_88 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6acf14d9d17be\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6acf14d9d17be\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-time-a-linux-program-cloudsavvy-it\/#Timing_Linux_Programs\" >Timing Linux Programs<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-time-a-linux-program-cloudsavvy-it\/#The_Linux_time_Tool\" >The Linux time Tool<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-time-a-linux-program-cloudsavvy-it\/#Using_time_For_Performance_Measuring\" >Using time For Performance Measuring<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-time-a-linux-program-cloudsavvy-it\/#Using_time_For_Code_Optimization\" >Using time For Code Optimization<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/how-to-time-a-linux-program-cloudsavvy-it\/#Wrapping_up\" >Wrapping up<\/a><\/li><\/ul><\/nav><\/div>\n<p><strong>&#8220;#How to Time a Linux Program \u2013 CloudSavvy IT&#8221;<\/strong><\/p>\n<div id=\"article-content-area\">\n<img loading=\"lazy\" decoding=\"async\" class=\"type:primaryImage alignnone size-full wp-image-4038\" data-pagespeed-lazy-src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2017\/07\/add8ac45.png?width=1198&amp;trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Bash Shell\" width=\"1400\" height=\"600\" src=\"\/pagespeed_static\/1.JiBnMqyl6S.gif\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Want to analyze how much wall clock time, kernel time, etc., a Linux program takes to run? Whether for performance testing, code optimization, or just <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a> curiosity, this quick guide will get you started!<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Timing_Linux_Programs\"><\/span>Timing Linux Programs<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Timing a Linux program helps one to understand how much time was spent. The versatile Linux <code>time<\/code> command can be used for this. The <code>time<\/code> command measures real (i.e. wall clock time), user, and sys time. User time is the time the program runs in user-mode, or in other words, outside of the kernel. The sys time is the time the program runs inside the kernel.<\/p>\n<p>It is important to note that both user time and sys time are actual CPU time spent inside user-mode and inside the kernel, respectively. In other words, when a program is blocked for a while and is not using the CPU, such time will not count towards <i>user<\/i> or <code>sys<\/code> times. Knowing this, we can accurately measure how much effective CPU time was used (by combining them).<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"The_Linux_time_Tool\"><\/span>The Linux <i>time<\/i> Tool<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Given that <i>user<\/i> and <code>sys<\/code> times report CPU time only, whereas <i>real<\/i> time reports actual wall clock time, it is (thus) very common to see the <code>time<\/code> tool return output where a combination of <i>user<\/i> + <i>sys<\/i> does not equal <i>real<\/i> time. An example can be seen when timing <code>sleep<\/code>:<\/p>\n<pre>time sleep 1&#13;\n<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-11955\" data-pagespeed-lazy-src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/06\/7d2045f2.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"A simple time command testing the time taken by a sleep 1 command at the Linux command prompt\" width=\"207\" height=\"130\" src=\"\/pagespeed_static\/1.JiBnMqyl6S.gif\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Here we timed the <code>sleep<\/code> command using the <code>time<\/code> tool. As we can see, our <i>real<\/i> time (1.001 seconds) matches our wall clock time, and requested time (<code>sleep 1<\/code> requests a sleep of one second) very well. We also see that extremely little CPU time had to be spent on the command as a whole: combining <i>user<\/i> + <i>sys<\/i> time, we see that only 0.001 seconds were spent.<\/p>\n<p>We can also, likely incorrectly, deduce that the kernel was not involved in this command, as the <i>sys<\/i> time is effectively 0. However, as the <code>time<\/code> manual states: \u201cWhen the running time of a command is very nearly zero, some values (e.g., the percentage of CPU used) may be reported as either zero (which is wrong) or a question mark.\u201d<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Using_time_For_Performance_Measuring\"><\/span>Using <i>time<\/i> For Performance Measuring<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>We can use <code>time<\/code> to evaluate how long given actions will take (i.e., wall clock time) and how much CPU time they consumed while doing so. As a simple example, we could evaluate if any file system cache is operating on our system. To do so, we could jump into the <code>\/usr<\/code> directory, which could easily hold 200k to 500k files on a commonplace Linux installation.<\/p>\n<p>Once there, we can use the <code>find<\/code> tool, timed by <code>time<\/code> to evaluate how long it would take to scan through all folders and list all files in the <code>\/usr<\/code> directory:<\/p>\n<pre>cd \/usr&#13;\ntime find . &gt;\/dev\/null 2&gt;&amp;1&#13;\n<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-11956\" data-pagespeed-lazy-src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/06\/161306a4.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Listing the contents of \/usr with find and timing the same\" width=\"354\" height=\"116\" src=\"\/pagespeed_static\/1.JiBnMqyl6S.gif\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>As we can see, it takes 12.484 seconds to list all files in the <code>\/usr<\/code> directory (and below it). We redirected the stdout output (standard output) of the command to <code>&gt;\/dev\/null<\/code> and also redirect any stderr errors (standard error) to <code>\/dev\/null<\/code> by using a redirect from stderr to stdout, i.e. <code>2&gt;&amp;1<\/code>.<\/p>\n<p>We also see that our CPU time is 1.043 seconds (user) + 2.908 seconds (sys) for a total of 3.951 seconds of CPU time.<\/p>\n<p>Let\u2019s test it another time by clearing our inode (and other) cache(s):<\/p>\n<pre>sync; echo 3 | sudo tee \/proc\/sys\/vm\/drop_caches&#13;\ncd \/usr&#13;\ntime find . &gt;\/dev\/null 2&gt;&amp;1&#13;\n<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-11957\" data-pagespeed-lazy-src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/06\/9c86482e.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Clearing the inode (and other) Linux caches and re-executing our find command in \/usr\" width=\"455\" height=\"152\" src=\"\/pagespeed_static\/1.JiBnMqyl6S.gif\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>The first command will drop the inodes cache, dentries (directory entries), and pagecache. This time the result came back a little quicker, with 1.255 seconds saved on the command. Likely a physical disk-based cache helped here.<\/p>\n<p>To demonstrate how well Linux caching works in general, let\u2019s re-run the command but this time without dropping the Linux caches:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-11958\" data-pagespeed-lazy-src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/06\/0a15afc4.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Re-executing the find command without clearing the caches results in much faster execution\" width=\"384\" height=\"136\" src=\"\/pagespeed_static\/1.JiBnMqyl6S.gif\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>Quite a difference! We see a large reduction in required time in all three timed areas and our command executes in less than half a second!<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Using_time_For_Code_Optimization\"><\/span>Using <i>time<\/i> For Code Optimization<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Once we feel comfortable using the <code>time<\/code> command at the command line, we can expand its use to optimize our Bash <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">script<\/a>s and code. For example, one commonly used approach amongst some professionals is to run a given command many times, like 1000 runs, and calculate the total (or average) time of these runs.<\/p>\n<p>Then, an alternative command can be used. That alternative command (or solution\/implementation \u2013 i.e., multiple commands taken together as a single piece of code to be timed) can then be timed again. In Linux (or more specifically in Bash coding etc.), there are often many ways to approach a given problem; there are usually multiple tools available to obtain\/achieve the same result.<\/p>\n<p>Testing which one performs best optimizes program run time and potentially other factors like disk I\/O (reducing disk wear) or memory utilization (allowing more programs to execute on the same instance). For optimizing the wall clock time, a certain tool uses, on average, as well as the CPU time consumed by the tool (another important optimization factor\/consideration) can be measured by the <code>time<\/code> tool.<\/p>\n<p>Let\u2019s explore a practical example of using the command line to execute a command we want to use in one of our scripts. The command will obtain a process list and display the second column. We use both <code>awk<\/code> and <code>sed<\/code> to do so, and run each command 1000 times to see the difference in performance overall.<\/p>\n<pre>time for ((i=1;i&lt;=1000;i++)); do ps -ef | awk '{print $2}' &gt;\/dev\/null 2&gt;&amp;1; done&#13;\ntime for ((i=1;i&lt;=1000;i++)); do ps -ef | sed 's|^[^ ]+[ t]+||;s|[ t].*||' &gt;\/dev\/null 2&gt;&amp;1; done&#13;\n<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-11959\" data-pagespeed-lazy-src=\"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2021\/06\/c17243d1.png?trim=1,1&amp;bg-color=000&amp;pad=1,1\" alt=\"Using time to analyze the average execution time of any command, allowing one to optimize Linux script runtimes\" width=\"915\" height=\"189\" src=\"\/pagespeed_static\/1.JiBnMqyl6S.gif\" onload=\"pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\" onerror=\"this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);\"\/><\/p>\n<p>While looking more complex (it uses a double regular expression to parse out the second column), our second command is a bit faster than our first command when it comes to wall clock time.<\/p>\n<p>Using a very similar setup (i.e. <code>time for ((i=1;i&lt;=1000;i++)); do command_to_be_timed &gt;\/dev\/null 2&gt;&amp;1; done<\/code> where command_to_be_timed is the command to be tested for wall clock or CPU time), one can time-test any command, or set of commands (as is the case here; we used both the <code>ps<\/code> and <code>awk<\/code>\/<code>sed<\/code> commands).<\/p>\n<p>Taking these steps for several time-consuming commands (in any Linux script) will help us to reduce the overall run time and\/or (if you optimize towards reduced CPU time) the system load of our scripts.<\/p>\n<p>If you want to learn more about regular expressions, How to Modify Text Using Regular Expressions With the sed Stream Editor is likely of interest.<\/p>\n<h2 role=\"heading\" aria-level=\"2\"><span class=\"ez-toc-section\" id=\"Wrapping_up\"><\/span>Wrapping up<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In this article, we explored the Linux <code>time<\/code> command. We clarified what <i>real<\/i>, <i>user<\/i> and <i>sys<\/i> times indicate and how the latter two relate to CPU usage. We also reviewed several examples of how to use <i>time<\/i> in practical ways.<\/p>\n<p>If you enjoyed reading this article, have a look at\u00a0Asserts, Errors, and Crashes: What\u2019s the Difference? and What is Stack Smashing? Can it be Fixed?.\n<\/p><\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/www.cloudsavvyit.com\/11954\/how-to-time-a-linux-program\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#How to Time a Linux Program \u2013 CloudSavvy IT&#8221; Want to analyze how much wall clock time, kernel time, etc., a Linux program takes to run? Whether for performance testing, code optimization, or just general curiosity, this quick guide will get you started! Timing Linux Programs Timing a Linux program helps one to understand how&#8230;<\/p>\n","protected":false},"author":1,"featured_media":291102,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/www.cloudsavvyit.com\/p\/uploads\/2017\/07\/add8ac45.png","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-291101","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/291101","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=291101"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/291101\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/291102"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=291101"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=291101"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=291101"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}