{"id":562605,"date":"2023-03-10T22:02:03","date_gmt":"2023-03-10T19:02:03","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/meta-semi-machine-learning-approach-outperforms-state-of-the-art-algorithms-in-deep-learning-tasks\/"},"modified":"2023-03-10T22:02:03","modified_gmt":"2023-03-10T19:02:03","slug":"meta-semi-machine-learning-approach-outperforms-state-of-the-art-algorithms-in-deep-learning-tasks","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/meta-semi-machine-learning-approach-outperforms-state-of-the-art-algorithms-in-deep-learning-tasks\/","title":{"rendered":"#&#8217;Meta-Semi&#8217; machine learning approach outperforms state-of-the-art algorithms in deep learning tasks"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_89 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6aef0c228b631\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6aef0c228b631\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/meta-semi-machine-learning-approach-outperforms-state-of-the-art-algorithms-in-deep-learning-tasks\/#%E2%80%9C%E2%80%98Meta-Semi_machine_learning_approach_outperforms_state-of-the-art_algorithms_in_deep_learning_tasks%E2%80%9D\" >&#8220;&#8216;Meta-Semi&#8217; machine learning approach outperforms state-of-the-art algorithms in deep learning tasks&#8221;<\/a><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"%E2%80%9C%E2%80%98Meta-Semi_machine_learning_approach_outperforms_state-of-the-art_algorithms_in_deep_learning_tasks%E2%80%9D\"><\/span>&#8220;&#8216;Meta-Semi&#8217; machine learning approach outperforms state-of-the-art algorithms in deep learning tasks&#8221;<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<div>\n<div class=\"article-gallery lightGallery\">\n<div data-thumb=\"https:\/\/scx1.b-cdn.net\/csz\/news\/tmb\/2023\/outperforms-state-of-t.jpg\" data-src=\"https:\/\/scx2.b-cdn.net\/gfx\/news\/hires\/2023\/outperforms-state-of-t.jpg\" data-sub-html=\"Meta-Semi trains deep networks using pseudo-labeled samples whose gradient directions are similar to labeled samples. Algorithm 1 shows the Meta Semi pseudo code. The Meta-Semi algorithm outperforms state-of-the-art semi-supervised learning algorithms. Credit: CAAI Artificial Intelligence Research, Tsinghua University Press\">\n<figure class=\"article-img\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/scx1.b-cdn.net\/csz\/news\/800a\/2023\/outperforms-state-of-t.jpg\" alt=\"Outperforms state-of-the-art algorithms in deep learning tasks\" title=\"Meta-Semi trains deep networks using pseudo-labeled samples whose gradient directions are similar to labeled samples. Algorithm 1 shows the Meta Semi pseudo code. The Meta-Semi algorithm outperforms state-of-the-art semi-supervised learning algorithms. Credit: CAAI Artificial Intelligence Research, Tsinghua University Press\" width=\"800\" height=\"343\"\/><figcaption class=\"text-darken text-low-up text-truncate-js text-truncate mt-3\">\n                Meta-Semi trains deep networks using pseudo-labeled samples whose gradient directions are similar to labeled samples. Algorithm 1 shows the Meta Semi pseudo code. The Meta-Semi algorithm outperforms state-of-the-art semi-supervised learning algorithms. Credit: CAAI Artificial Intelligence Research, Tsinghua University Press<br \/>\n            <\/figcaption><\/figure>\n<\/div>\n<\/div>\n<p>Deep learning based semi-supervised learning algorithms have shown promising results in recent years. However, they are not yet practical in real semi-supervised learning scenarios, such as medical image processing, hyper-spectral image classification, network traffic recognition, and document recognition.<\/p>\n<p>In these types of scenarios, the labeled data is scarce for hyper-parameter search, because they introduce multiple tunable hyper-parameters. A research team has proposed a novel meta-learning based semi-supervised learning algorithm called Meta-Semi, that requires tuning only one additional hyper-parameter. Their Meta-Semi <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>roach outperforms state-of-the-art semi-supervised learning algorithms.<\/p>\n<p>The team published their work in the journal <i>CAAI Artificial Intelligence Research<\/i>.<\/p>\n<p>Deep learning, a machine learning technique where computers learn by example, is showing success in supervised tasks. However, the process of data labeling, where the raw data is identified and labeled, is time-consuming and costly. Deep learning in supervised tasks can be successful when there is plenty of annotated training data available. Yet in many real-world applications, only a small subset of all the available training data are associated with labels.<\/p>\n<p>&#8220;The recent success of deep learning in supervised tasks is fueled by abundant annotated training data,&#8221; said Gao Huang, associate professor with the Department of Automation at Tsinghua University. However, the time-consuming, costly collection of precise labels is a challenge researchers have to overcome. &#8220;Meta-semi, as a state-of-the-art semi-supervised learning approach, can effectively train deep models with a small number of labeled samples,&#8221; said Huang.<\/p>\n<p>With the research team&#8217;s Meta-Semi classification algorithm, they efficiently exploit the labeled data, while requiring only one additional hyper-parameter to achieve impressive performance under various conditions. In machine learning, a hyper-parameter is a parameter whose value can be used to direct the learning process.<\/p>\n<p>&#8220;Most deep learning based semi-supervised learning algorithms introduce multiple tunable hyper-parameters, making them less practical in real semi-supervised learning scenarios where the labeled data is scarce for extensive hyper-parameter search,&#8221; said Huang.<\/p>\n<p>The team developed their algorithm working from the assumption that the network could be trained effectively with the correctly pseudo-labeled unannotated samples. First they generated soft pseudo labels for the unlabeled data online during the training process based on the network predictions.<\/p>\n<p>Then they filtered out the samples whose pseudo labels were incorrect or unreliable and trained the model using the remaining data with relatively reliable pseudo labels. Their process naturally yielded a meta-learning formulation where the correctly pseudo-labeled data had a similar distribution to the labeled data. In their process, if the network is trained with the pseudo-labeled data, the final loss on the labeled data should be minimized as well.<\/p>\n<p>The team&#8217;s Meta-Semi algorithm achieved competitive performance under various conditions of semi-supervised learning. &#8220;Empirically, Meta-Semi outperforms state-of-the-art semi-supervised learning algorithms significantly on the challenging semi-supervised CIFAR-100 and STL-10 tasks, and achieves competitive performance on CIFAR-10 and SVHN,&#8221; said Huang.<\/p>\n<p>CIFAR-10, STL-10, and SVHN are datasets, or collections of images, that are frequently used in training machine learning algorithms. &#8220;We also show theoretically that Meta-Semi converges to the stationary point of the loss function on labeled data under mild conditions,&#8221; said Huang. Compared to existing deep semi-supervised learning algorithms, Meta-Semi requires much less effort for tuning hyper-parameters, but achieves state-of-the-art performance on the four competitive datasets.<\/p>\n<p>Looking ahead to future work, the research team&#8217;s aim is to develop an effective, practical and robust semi-supervised learning algorithm. &#8220;The algorithm should require a minimal number of data annotations, minimal efforts of hyper-parameter tuning, and a minimized training time. To attain this goal, our future work may focus on reducing the training cost of Meta-Semi,&#8221; said Huang.\n                                                                                                                            <\/p>\n<div class=\"article-main__more p-4\">\n                                                                                                <strong>More information:<\/strong><br \/>\n                                                Yulin Wang et al, Meta-Semi: A Meta-Learning Approach for Semi-Supervised Learning, <i>CAAI Artificial Intelligence Research<\/i> (2023).  <a rel=\"nofollow noopener\" target=\"_blank\" data-doi=\"1\" href=\"https:\/\/dx.doi.org\/10.26599\/AIR.2022.9150011\">DOI: 10.26599\/AIR.2022.9150011<\/a><\/p>\n<\/div>\n<p>\n                                                Provided by<br \/>\n                                                                                                    Tsinghua University Press<\/p>\n<p>                                        <!-- print only --><\/p>\n<div class=\"d-none d-print-block\">\n<p>                                                <strong>Citation<\/strong>:<br \/>\n                                                &#8216;Meta-Semi&#8217; machine learning approach outperforms state-of-the-art algorithms in deep learning tasks (2023, March 10)<br \/>\n                                                retrieved 11 March 2023<br \/>\n                                                from https:\/\/techxplore.com\/<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>\/2023-03-meta-semi-machine-approach-outperforms-state-of-the-art.html<\/p>\n<p>                                            This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no<br \/>\n                                            part may be reproduced without the written permission. The content is provided for information purposes only.<\/p><\/div>\n<\/p><\/div>\n<p><script id=\"facebook-jssdk\" async=\"\" src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong>\n<\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more Like this articles, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/science\/\" target=\"_blank\" rel=\"noopener\">Science category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/techxplore.com\/news\/2023-03-meta-semi-machine-approach-outperforms-state-of-the-art.html\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;&#8216;Meta-Semi&#8217; machine learning approach outperforms state-of-the-art algorithms in deep learning tasks&#8221; Meta-Semi trains deep networks using pseudo-labeled samples whose gradient directions are similar to labeled samples. Algorithm 1 shows the Meta Semi pseudo code. The Meta-Semi algorithm outperforms state-of-the-art semi-supervised learning algorithms. Credit: CAAI Artificial Intelligence Research, Tsinghua University Press Deep learning based semi-supervised learning&#8230;<\/p>\n","protected":false},"author":1,"featured_media":562606,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/scx2.b-cdn.net\/gfx\/news\/hires\/2023\/outperforms-state-of-t.jpg","fifu_image_alt":"","footnotes":""},"categories":[16],"tags":[],"class_list":["post-562605","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sciencee"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/562605","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=562605"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/562605\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/562606"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=562605"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=562605"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=562605"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}