{"id":82707,"date":"2020-10-06T10:59:10","date_gmt":"2020-10-06T07:59:10","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research\/"},"modified":"2020-10-06T10:59:10","modified_gmt":"2020-10-06T07:59:10","slug":"how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research\/","title":{"rendered":"#How \u2018less-than-one-shot learning\u2019 could open up new venues for machine learning research"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a321a0475287\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a321a0475287\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research\/#The_k-NN_classifier\" >The k-NN classifier<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research\/#Machine_learning_with_less_than_one_example_per_class\" >Machine learning with less than one example per class<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research\/#New_venues_for_machine_learning_research\" >New venues for machine learning research<\/a><\/li><\/ul><\/nav><\/div>\n<p>&#8220;<strong>#How \u2018less-than-one-shot learning\u2019 could open up new venues for machine learning research<\/strong>&#8221;<\/p>\n<div>\n                                If I told you to imagine something between a horse and a bird\u2014say, a flying horse\u2014would you need to see a concrete example? Such a creature does not exist, but nothing prevents us from using our imagination to create one: the Pegasus.<\/p>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-8407 jetpack-lazy-image jetpack-lazy-image--handled\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=696%2C162&amp;ssl=1\" sizes=\"auto, (max-width: 696px) 100vw, 696px\" srcset=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=1024%2C238&amp;ssl=1 1024w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=300%2C70&amp;ssl=1 300w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=768%2C178&amp;ssl=1 768w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=1536%2C356&amp;ssl=1 1536w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=2048%2C475&amp;ssl=1 2048w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=696%2C161&amp;ssl=1 696w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=1068%2C248&amp;ssl=1 1068w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=1810%2C420&amp;ssl=1 1810w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?resize=1920%2C446&amp;ssl=1 1920w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?w=1392&amp;ssl=1 1392w\" alt=\"Pegasus\" width=\"696\" height=\"162\" data-attachment-id=\"8407\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/10\/01\/less-than-one-shot-machine-learning\/pegasus\/\" data-orig-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?fit=2560%2C594&amp;ssl=1\" data-orig-size=\"2560,594\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"Pegasus\" data-image-description=\"\" data-medium-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?fit=300%2C70&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/Pegasus.jpg?fit=696%2C162&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\"\/><\/figure>\n<p>The human mind has all kinds of mechanisms to create new concepts by combining abstract and concrete knowledge it has of the real world. We can imagine existing things that we might have never seen (a horse with a long neck \u2014 a giraffe), as well as things that do not exist in real life (a winged serpent that breathes fire \u2014 a dragon). This cognitive flexibility allows us to learn new things with few and sometimes no new examples.<\/p>\n<p>In contrast, <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2017\/08\/28\/artificial-intelligence-machine-learning-deep-learning\/\">machine learning<\/a> and <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2019\/02\/15\/what-is-deep-learning-neural-networks\/\">deep learning<\/a>, the current leading fields of artificial intelligence, are known to require many examples to learn new tasks, even when they are related to things they already know.<\/p>\n<p>Overcoming this challenge has led to a host of research work and innovation in machine learning. And although we are still far from creating artificial intelligence that can replicate <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/07\/13\/ai-barrier-meaning-understanding\/\">the brain\u2019s capacity for understanding<\/a>, the progress in the field is remarkable.<\/p>\n<p>For instance, <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2019\/06\/10\/what-is-transfer-learning\/\">transfer learning<\/a> is a technique that enables developers to finetune an artificial neural network for a new task without the need for many training examples. Few-shot and <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/08\/12\/what-is-one-shot-learning\/\">one-shot learning<\/a> enable a machine learning model trained on one task to perform a related task with a single or very few new examples. For instance, if you have an image classifier trained to detect volleyballs and soccer balls, you can use one-shot learning to add basketball to the list of classes it can detect.<\/p>\n<p><em>[Read: A beginner\u2019s guide to the math that powers machine learning]<\/em><\/p>\n<p>A new technique dubbed \u201cless-than-one-shot learning\u201d (or LO-shot learning), recently developed by AI scientists at the University of Waterloo, takes one-shot learning to the next level. The idea behind LO-shot learning is that to train a machine learning model to detect M classes, you need less than one sample per class. The technique, introduced in <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/2009.08449\">a paper published in the arXiv preprocessor<\/a>, is still in its early stages but shows promise and can be useful in various scenarios where there is not enough data or too many classes.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_k-NN_classifier\"><\/span>The k-NN classifier<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<figure class=\"wp-block-image size-large\">\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"jetpack-lazy-image jetpack-lazy-image--handled wp-image-8402 lazy\" sizes=\"auto, (max-width: 696px) 100vw, 696px\" alt=\"k-NN machine learning algorithm\" width=\"696\" height=\"481\" data-attachment-id=\"8402\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/10\/01\/less-than-one-shot-machine-learning\/k-nn-machine-learning-algorithm\/\" data-orig-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?fit=1654%2C1144&amp;ssl=1\" data-orig-size=\"1654,1144\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"k-NN machine learning algorithm\" data-image-description=\"\" data-medium-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?fit=300%2C207&amp;ssl=1\" data-large-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?fit=696%2C481&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\" src=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=696%2C481&amp;ssl=1\" data-lazy=\"true\" srcset=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=1024%2C708&amp;ssl=1 1024w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=300%2C207&amp;ssl=1 300w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=768%2C531&amp;ssl=1 768w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=1536%2C1062&amp;ssl=1 1536w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=100%2C70&amp;ssl=1 100w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=218%2C150&amp;ssl=1 218w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=696%2C481&amp;ssl=1 696w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=1068%2C739&amp;ssl=1 1068w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?resize=607%2C420&amp;ssl=1 607w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?w=1654&amp;ssl=1 1654w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/k-NN-machine-learning-algorithm.png?w=1392&amp;ssl=1 1392w\"\/><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/10\/06\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research-syndication\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F10%2F06%2Fhow-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: The k-NN machine learning algorithm classifies data by finding the closest instances.\" data-title=\"Share The k-NN machine learning algorithm classifies data by finding the closest instances. on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share The k-NN machine learning algorithm classifies data by finding the closest instances. on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>The k-NN machine learning algorithm classifies data by finding the closest instances.<\/figcaption><\/figure>\n<\/p>\n<\/figure>\n<p>The LO-shot learning technique proposed by the researchers <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lies to the \u201ck-nearest neighbors\u201d machine learning algorithm. K-NN can be used for both classification (determining the category of an input) or regression (predicting the outcome of an input) tasks. But for the sake of this discussion, we\u2019ll still to classification.<\/p>\n<p>As the name implies, k-NN classifies input data by comparing it to its <em>k <\/em>nearest neighbors (<em>k <\/em>is an adjustable parameter). Say you want to create a k-NN machine learning model that classifies hand-written digits. First you provide it with a set of labeled images of digits. Then, when you provide the model with a new, unlabeled image, it will determine its class by looking at its nearest neighbors.<\/p>\n<p>For instance, if you set <em>k <\/em>to 5, the machine learning model will find the five most similar digit photos for each new input. If, say three of them belong to the class \u201c7,\u201d it will classify the image as the digit seven.<\/p>\n<p>k-NN is an \u201cinstance-based\u201d machine learning algorithm. As you provide it with more labeled examples of each class, its accuracy improves but its performance degrades, because each new sample adds new comparisons operations.<br \/>\n<iframe loading=\"lazy\" title=\"How kNN algorithm works\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/UqYde-LULfs?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><br \/>\nIn their LO-shot learning paper, the researchers showed that you can achieve accurate results with k-NN while providing fewer examples than there are classes. \u201cWe propose \u2018less than one\u2019-shot learning (LO-shot learning), a setting where a model must learn <em>N<\/em> new classes given only <em>M &lt; N<\/em> examples, less than one example per class,\u201d the AI researchers write. \u201cAt first glance, this appears to be an impossible task, but we both theoretically and empirically demonstrate feasibility.\u201d<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Machine_learning_with_less_than_one_example_per_class\"><\/span>Machine learning with less than one example per class<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The classic k-NN algorithm provides \u201chard labels,\u201d which means for every input, it provides exactly one class to which it belongs. Soft labels, on the other hand, provide the probability that an input belongs to each of the output classes (e.g., there\u2019s a 20% chance it\u2019s a \u201c2\u201d, 70% chance it\u2019s a \u201c5,\u201d and a 10% chance it\u2019s a \u201c3\u201d).<\/p>\n<p>In their work, the AI researchers at the University of Waterloo explored whether they could use soft labels to <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a>ize the capabilities of the k-NN algorithm. The proposition of LO-shot learning is that soft label prototypes should allow the machine learning model to classify <em>N <\/em>classes with less than <em>N <\/em>labeled instances.<\/p>\n<p>The technique builds on previous work the researchers had done on <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/1910.02551\">soft labels and data distillation<\/a>. \u201cDataset distillation is a process for producing small synthetic datasets that train models to the same accuracy as training them on the full training set,\u201d Ilia Sucholutsky, co-author of the paper, told <em>TechTalks<\/em>. \u201cBefore soft labels, dataset distillation was able to represent datasets like MNIST using as few as one example per class. I realized that adding soft labels meant I could actually represent MNIST using less than one example per class.\u201d<\/p>\n<p><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"http:\/\/yann.lecun.com\/exdb\/mnist\/\">MNIST<\/a> is a database of images of handwritten digits often used in training and testing machine learning models. Sucholutsky and his colleague Matthias Schonlau managed to achieve above-90 percent accuracy on MNIST with just five synthetic examples on the <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/01\/06\/convolutional-neural-networks-cnn-convnets\/\">convolutional neural network<\/a> LeNet.<\/p>\n<p>\u201cThat result really surprised me, and it\u2019s what got me thinking more broadly about this LO-shot learning setting,\u201d Sucholutsky said.<\/p>\n<p>Basically, LO-shot uses soft labels to create new classes by partitioning the space between existing classes.<\/p>\n<figure class=\"wp-block-image size-large\">\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"jetpack-lazy-image jetpack-lazy-image--handled wp-image-8399 lazy\" sizes=\"auto, (max-width: 696px) 100vw, 696px\" alt=\"less-than-one-shot learning example with two classes\" width=\"696\" height=\"514\" data-attachment-id=\"8399\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/10\/01\/less-than-one-shot-machine-learning\/less-than-one-shot-learning-example-with-two-classes\/\" data-orig-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?fit=1338%2C988&amp;ssl=1\" data-orig-size=\"1338,988\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"less-than-one-shot learning example with two classes\" data-image-description=\"\" data-medium-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?fit=300%2C222&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?fit=696%2C514&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?resize=696%2C514&amp;ssl=1\" data-lazy=\"true\" srcset=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?resize=1024%2C756&amp;ssl=1 1024w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?resize=300%2C222&amp;ssl=1 300w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?resize=768%2C567&amp;ssl=1 768w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?resize=80%2C60&amp;ssl=1 80w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?resize=696%2C514&amp;ssl=1 696w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?resize=1068%2C789&amp;ssl=1 1068w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?resize=569%2C420&amp;ssl=1 569w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-example-with-two-classes.png?w=1338&amp;ssl=1 1338w\"\/><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/10\/06\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research-syndication\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F10%2F06%2Fhow-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: LO-shot learning uses soft labels to partition the space between existing classes.\" data-title=\"Share LO-shot learning uses soft labels to partition the space between existing classes. on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share LO-shot learning uses soft labels to partition the space between existing classes. on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>LO-shot learning uses soft labels to partition the space between existing classes.<\/figcaption><\/figure>\n<\/p>\n<\/figure>\n<p>In the example above, there are two instances to tune the machine learning model (shown with black dots). A classic k-NN algorithm would split the space between the two dots between the two classes. But the \u201csoft-label prototype k-NN\u201d (SLaPkNN) algorithm, as the OL-shot learning model is called, creates a new space between the two classes (the green area), which represents a new label (think horse with wings). Here we have achieved <em>N <\/em>classes with <em>N-1<\/em> samples.<\/p>\n<p>In the paper, the researchers show that LO-shot learning can be scaled up to detect <em>3N-2<\/em> classes using <em>N<\/em> labels and even beyond.<\/p>\n<figure class=\"wp-block-image size-large\">\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"jetpack-lazy-image jetpack-lazy-image--handled wp-image-8401 lazy\" alt=\"less-than-one-shot learning examples\" width=\"696\" height=\"260\" data-attachment-id=\"8401\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/10\/01\/less-than-one-shot-machine-learning\/less-than-one-shot-learning-examples\/\" data-orig-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-examples.png?fit=1440%2C538&amp;ssl=1\" data-orig-size=\"1440,538\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"less-than-one-shot learning examples\" data-image-description=\"\" data-medium-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-examples.png?fit=300%2C112&amp;ssl=1\" data-large-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-examples.png?fit=696%2C260&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/less-than-one-shot-learning-examples-1024x383.png?resize=696%2C260&amp;ssl=1\" data-lazy=\"true\"\/><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/10\/06\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research-syndication\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F10%2F06%2Fhow-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: LO-shot learning can be extended to obtain multiple classes per instance. Left: 10 classes obtained from four instances. Right: 13 classes obtained from five instances.\" data-title=\"Share LO-shot learning can be extended to obtain multiple classes per instance. Left: 10 classes obtained from four instances. Right: 13 classes obtained from five instances. on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share LO-shot learning can be extended to obtain multiple classes per instance. Left: 10 classes obtained from four instances. Right: 13 classes obtained from five instances. on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>LO-shot learning can be extended to obtain multiple classes per instance. Left: 10 classes obtained from four instances. Right: 13 classes obtained from five instances.<\/figcaption><\/figure>\n<\/p>\n<\/figure>\n<p>In their experiments, Sucholutsky and Schonlau found that with the right configurations for the soft labels, LO-shot machine learning can provide reliable results even when you have noisy data.<\/p>\n<p>\u201cI think LO-shot learning can be made to work from other sources of information as well\u2014similar to how many zero-shot learning methods do\u2014but soft labels are the most straightforward approach,\u201d Sucholutsky said, adding that there are already several methods that can find the right soft labels for LO-shot machine learning.<\/p>\n<p>While the paper displays the power of LO-shot learning with the k-NN classifier, Sucholutsky says the technique applies to other machine learning algorithms as well. \u201cThe analysis in the paper focuses specifically on k-NN just because it\u2019s easier to analyze, but it should work for any classification model that can make use of soft labels,\u201d Sucholutsky said. The researchers will soon release a more comprehensive paper that shows the application of LO-shot learning to deep learning models.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"New_venues_for_machine_learning_research\"><\/span>New venues for machine learning research<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-8405 jetpack-lazy-image jetpack-lazy-image--handled\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?resize=696%2C426&amp;ssl=1\" sizes=\"auto, (max-width: 696px) 100vw, 696px\" srcset=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?resize=1024%2C627&amp;ssl=1 1024w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?resize=300%2C184&amp;ssl=1 300w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?resize=768%2C470&amp;ssl=1 768w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?resize=696%2C426&amp;ssl=1 696w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?resize=1068%2C654&amp;ssl=1 1068w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?resize=686%2C420&amp;ssl=1 686w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?w=1432&amp;ssl=1 1432w\" alt=\"3d objects\" width=\"696\" height=\"426\" data-attachment-id=\"8405\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/10\/01\/less-than-one-shot-machine-learning\/3d-objects\/\" data-orig-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?fit=1432%2C877&amp;ssl=1\" data-orig-size=\"1432,877\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"3d objects\" data-image-description=\"\" data-medium-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?fit=300%2C184&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/10\/3d-objects.jpg?fit=696%2C426&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\"\/><\/figure>\n<p>\u201cFor instance-based algorithms like k-NN, the efficiency improvement of LO-shot learning is quite large, especially for datasets with a large number of classes,\u201d Susholutsky said. \u201cMore broadly, LO-shot learning is useful in any kind of setting where a classification algorithm is applied to a dataset with a large number of classes, especially if there are few, or no, examples available for some classes. Basically, most settings where zero-shot learning or few-shot learning are useful, LO-shot learning can also be useful.\u201d<\/p>\n<p>For instance, a <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2019\/01\/14\/what-is-computer-vision\/\">computer vision<\/a> system that must identify thousands of objects from images and video frames can benefit from this machine learning technique, especially if there are no examples available for some of the objects. Another application would be to tasks that naturally have soft-label information, like <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2018\/02\/20\/ai-machine-learning-nlg-nlp\/\">natural language processing<\/a> systems that perform sentiment analysis (e.g., a sentence can be both sad and angry simultaneously).<\/p>\n<p>In their paper, the researchers describe \u201cless than one\u201d-shot learning as \u201ca viable new direction in machine learning research.\u201d<\/p>\n<p>\u201cWe believe that creating a soft-label prototype generation algorithm that specifically optimizes prototypes for LO-shot learning is an important next step in exploring this area,\u201d they write.<\/p>\n<p>\u201cSoft labels have been explored in several settings before. What\u2019s new here is the extreme setting in which we explore them,\u201d Susholutsky said.\u00a0\u201cI think it just wasn\u2019t a directly obvious idea that there is another regime hiding between one-shot and zero-shot learning.\u201d<\/p>\n<p><i><span>This article was originally published by\u00a0<\/span><\/i><i><span><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" class=\"author url fn\" title=\"Posts by Ben Dickson\" href=\"https:\/\/bdtechtalks.com\/author\/bendee983\/\">Ben Dickson<\/a> on <\/span><\/i><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/\"><i><span>TechTalks<\/span><\/i><\/a><i><span>, a publication that examines trends in <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" data-internallinksmanager029f6b8e52c=\"4\" title=\"Technology\" target=\"_blank\" rel=\"noopener\">technology<\/a>, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech and what we need to look out for. You can read the original article <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/10\/01\/less-than-one-shot-machine-learning\/\">here<\/a>.<br \/><\/span><\/i><\/p>\n<p class=\"c-post-pubDate\">\n                                    Published October 6, 2020 \u2014 07:59 UTC<\/p><\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><script data-src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js#xfbml=1&amp;appId=378011798897423&amp;version=v2.6\" id=\"socialSrcFacebook\" type=\"text\/template\"><\/script><\/p>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener noreferrer\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/neural\/2020\/10\/06\/how-less-than-one-shot-learning-could-open-up-new-venues-for-machine-learning-research-syndication\/\" target=\"_blank\" rel=\"noopener noreferrer\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#How \u2018less-than-one-shot learning\u2019 could open up new venues for machine learning research&#8221; If I told you to imagine something between a horse and a bird\u2014say, a flying horse\u2014would you need to see a concrete example? Such a creature does not exist, but nothing prevents us from using our imagination to create one: the Pegasus. The&#8230;<\/p>\n","protected":false},"author":1,"featured_media":82708,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/1.jpg&signature=ff645e00a7ae08fbc8387f469b724cf5","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-82707","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/82707","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=82707"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/82707\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/82708"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=82707"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=82707"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=82707"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}