{"id":437172,"date":"2022-04-25T18:22:29","date_gmt":"2022-04-25T15:22:29","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/an-easier-way-to-teach-robots-new-skills\/"},"modified":"2022-04-25T18:22:29","modified_gmt":"2022-04-25T15:22:29","slug":"an-easier-way-to-teach-robots-new-skills","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/an-easier-way-to-teach-robots-new-skills\/","title":{"rendered":"#An easier way to teach robots new skills"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a42551608a13\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a42551608a13\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/an-easier-way-to-teach-robots-new-skills\/#%E2%80%9CAn_easier_way_to_teach_robots_new_skills%E2%80%9D\" >&#8220;An easier way to teach robots new skills&#8221;<\/a><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"%E2%80%9CAn_easier_way_to_teach_robots_new_skills%E2%80%9D\"><\/span>&#8220;An easier way to teach robots new skills&#8221;<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<div>\n<div class=\"article-gallery lightGallery\">\n<div data-thumb=\"https:\/\/scx1.b-cdn.net\/csz\/news\/tmb\/2022\/an-easier-way-to-teach.jpg\" data-src=\"https:\/\/scx2.b-cdn.net\/gfx\/news\/2022\/an-easier-way-to-teach.jpg\" data-sub-html=\"MIT researchers have developed a system that enables a robot to learn a new pick-and-place task based on only a handful of human examples. This could allow a human to reprogram a robot to grasp never-before-seen objects, presented in random poses, in about 15 minutes. Credit: Massachusetts Institute of Technology\">\n<figure class=\"article-img\">\n            <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/scx1.b-cdn.net\/csz\/news\/800a\/2022\/an-easier-way-to-teach.jpg\" alt=\"An easier way to teach robots new skills\" title=\"MIT researchers have developed a system that enables a robot to learn a new pick-and-place task based on only a handful of human examples. This could allow a human to reprogram a robot to grasp never-before-seen objects, presented in random poses, in about 15 minutes. Credit: Massachusetts Institute of Technology\" width=\"800\" height=\"530\"\/><figcaption class=\"text-darken text-low-up text-truncate-js text-truncate mt-3\">\n                MIT researchers have developed a system that enables a robot to learn a new pick-and-place task based on only a handful of human examples. This could allow a human to reprogram a robot to grasp never-before-seen objects, presented in random poses, in about 15 minutes. Credit: Massachusetts Institute of <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" data-internallinksmanager029f6b8e52c=\"4\" title=\"Technology\" target=\"_blank\" rel=\"noopener\">Technology<\/a><br \/>\n            <\/figcaption><\/figure>\n<\/p><\/div>\n<\/div>\n<p>With e-commerce orders pouring in, a warehouse robot picks mugs off a shelf and places them into boxes for shipping. Everything is humming along, until the warehouse processes a change and the robot must now grasp taller, narrower mugs that are stored upside down.<\/p>\n<p>                                                                                Reprogramming that robot involves hand-labeling thousands of images that show it how to grasp these new mugs, then training the system all over again.<\/p>\n<p>But a new technique developed by MIT researchers would require only a handful of human demonstrations to reprogram the robot. This machine-learning method enables a robot to pick up and place never-before-seen objects that are in random poses it has never encountered. Within 10 to 15 minutes, the robot would be ready to perform a new pick-and-place task.<\/p>\n<p>The technique uses a neural network specifically designed to reconstruct the shapes of 3D objects. With just a few demonstrations, the system uses what the neural network has learned about 3D geometry to grasp new objects that are similar to those in the demos.<\/p>\n<p>In simulations and using a real robotic arm, the researchers show that their system can effectively manipulate never-before-seen mugs, bowls, and bottles, arranged in random poses, using only 10 demonstrations to teach the robot.<\/p>\n<p>&#8220;Our major contribution is the <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a> ability to much more efficiently provide new skills to robots that need to operate in more unstructured environments where there could be a lot of variability. The concept of generalization by construction is a fascinating capability because this problem is typically so much harder,&#8221; says Anthony Simeonov, a graduate student in electrical engineering and computer <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/sciencee\/\" data-internallinksmanager029f6b8e52c=\"5\" title=\"Science\" target=\"_blank\" rel=\"noopener\">science<\/a> (EECS) and co-lead author of the paper.<\/p>\n<p>Simeonov wrote the paper with co-lead author Yilun Du, an EECS graduate student; Andrea Tagliasacchi, a staff research scientist at Google Brain; Joshua B. Tenenbaum, the Paul E. Newton Career Development Professor of Cognitive Science and Computation in the Department of Brain and Cognitive Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); Alberto Rodriguez, the Class of 1957 Associate Professor in the Department of Mechanical Engineering; and senior authors Pulkit Agrawal, a professor in CSAIL, and Vincent Sitzmann, an incoming assistant professor in EECS. The research will be presented at the International Conference on Robotics and Automation.<br \/>\n                                            <!-- Google middle Adsense block --><\/p>\n<p><b>Grasping geometry<\/b><\/p>\n<p>A robot may be trained to pick up a specific item, but if that object is lying on its side (perhaps it fell over), the robot sees this as a completely new scenario. This is one reason it is so hard for machine-learning systems to generalize to new object orientations.<\/p>\n<p>To overcome this challenge, the researchers created a new type of neural network model, a Neural De<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">script<\/a>or Field (NDF), that learns the 3D geometry of a class of items. The model computes the geometric representation for a specific item using a 3D point cloud, which is a set of data points or coordinates in three dimensions. The data points can be obtained from a depth camera that provides information on the distance between the object and a viewpoint. While the network was trained in simulation on a large dataset of synthetic 3D shapes, it can be directly applied to objects in the real world.<\/p>\n<p>The team designed the NDF with a property known as equivariance. With this property, if the model is shown an image of an upright mug, and then shown an image of the same mug on its side, it understands that the second mug is the same object, just rotated.<\/p>\n<figure class=\"mb-4\" itemscope=\"\" itemtype=\"http:\/\/schema.org\/VideoObject\">\n    <meta itemprop=\"name\" content=\"An easier way to teach robots new skills\"\/><br \/>\n    <meta itemprop=\"url\" content=\"https:\/\/www.youtube.com\/watch\/?v=dXl9xI2LrRw\"\/><br \/>\n    <meta itemprop=\"description\" content=\"Credit: Massachusetts Institute of Technology\"\/><br \/>\n    <meta itemprop=\"uploadDate\" content=\"2022-04-25T07:24:23-04:00\"\/><br \/>\n        <meta itemprop=\"embedUrl\" content=\"https:\/\/www.youtube.com\/embed\/dXl9xI2LrRw\"\/><br \/>\n    <meta itemprop=\"thumbnailUrl\" content=\"https:\/\/img.youtube.com\/vi\/dXl9xI2LrRw\/maxresdefault.jpg\"\/><\/p>\n<p>             <iframe loading=\"lazy\" title=\"Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/dXl9xI2LrRw?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><figcaption class=\"text-darken text-low-up mt-4\" itemprop=\"caption\">Credit: Massachusetts Institute of Technology<\/figcaption><\/figure>\n<p>&#8220;This equivariance is what allows us to much more effectively handle cases where the object you observe is in some arbitrary orientation,&#8221; Simeonov says.<\/p>\n<p>As the NDF learns to reconstruct shapes of similar objects, it also learns to associate related parts of those objects. For instance, it learns that the handles of mugs are similar, even if some mugs are taller or wider than others, or have smaller or longer handles.<\/p>\n<p>&#8220;If you wanted to do this with another approach, you&#8217;d have to hand-label all the parts. Instead, our approach automatically discovers these parts from the shape reconstruction,&#8221; Du says.<\/p>\n<p>The researchers use this trained NDF model to teach a robot a new skill with only a few physical examples. They move the hand of the robot onto the part of an object they want it to grip, like the rim of a bowl or the handle of a mug, and record the locations of the fingertips.<\/p>\n<p>Because the NDF has learned so much about 3D geometry and how to reconstruct shapes, it can infer the structure of a new shape, which enables the system to transfer the demonstrations to new objects in arbitrary poses, Du explains.<\/p>\n<p><b>Picking a winner<\/b><\/p>\n<p>They tested their model in simulations and on a real robotic arm using mugs, bowls, and bottles as objects. Their method had a success rate of 85 percent on pick-and-place tasks with new objects in new orientations, while the best baseline was only able to achieve a success rate of 45 percent. Success means grasping a new object and placing it on a target location, like hanging mugs on a rack.<\/p>\n<p>Many baselines use 2D image information rather than 3D geometry, which makes it more difficult for these methods to integrate equivariance. This is one reason the NDF technique performed so much better.<\/p>\n<p>While the researchers were happy with its performance, their method only works for the particular object category on which it is trained. A robot taught to pick up mugs won&#8217;t be able to pick up boxes or headphones, since these objects have geometric features that are too different than what the network was trained on.<\/p>\n<p>&#8220;In the future, scaling it up to many categories or completely letting go of the notion of category altogether would be ideal,&#8221; Simeonov says.<\/p>\n<p>They also plan to adapt the system for nonrigid objects and, in the longer term, enable the system to perform pick-and-place tasks when the target area changes.\n                                                                                                                        <\/p>\n<hr\/>\n<div class=\"article-main__explore my-4 d-print-none\">\n<p>                                            New method allows robot vision to identify occluded objects\n                                        <\/p><\/div>\n<hr class=\"mb-4\"\/>\n<div class=\"article-main__more p-4\">\n                                                                                                <strong>More information:<\/strong><br \/>\n                                                Anthony Simeonov et al, Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation. arXiv:2112.05124v1 [cs.RO], <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/doi.org\/10.48550\/arXiv.2112.05124\">doi.org\/10.48550\/arXiv.2112.05124<\/a><\/p><\/div>\n<div class=\"d-inline-block text-medium my-4\">\n                                                Provided by<br \/>\n                                                                                                    Massachusetts Institute of Technology<br \/>\n                                                                                                        <a rel=\"nofollow noopener\" target=\"_blank\" class=\"icon_open\" href=\"http:\/\/web.mit.edu\/\"><br \/>\n                                                        <svg>\n                                                            <use href=\"https:\/\/techx.b-cdn.net\/tmpl\/v2\/img\/svg\/sprite.svg#icon_open\" x=\"0\" y=\"0\"\/>\n                                                        <\/svg><br \/>\n                                                    <\/a><\/p><\/div>\n<p class=\"article-main__note mt-4\">\n                                                <i>This story is republished courtesy of MIT <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">News<\/a> (<a rel=\"nofollow noopener\" target=\"_blank\" href=\"http:\/\/web.mit.edu\/newsoffice\/\">web.mit.edu\/newsoffice\/<\/a>), a popular site that covers news about MIT research, innovation and teaching.<\/i><\/p>\n<p>                                        <!-- print only --><\/p>\n<div class=\"d-none d-print-block\">\n<p>                                                 <strong>Citation<\/strong>:<br \/>\n                                                 An easier way to teach robots new skills (2022, April 25)<br \/>\n                                                 retrieved 25 April 2022<br \/>\n                                                 from https:\/\/techxplore.com\/news\/2022-04-easier-robots-skills.html<\/p>\n<p>                                            This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no<br \/>\n                                            part may be reproduced without the written permission. The content is provided for information purposes only.<\/p><\/div>\n<\/p><\/div>\n<p><script id=\"facebook-jssdk\" async=\"\" src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more Like this articles, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/science\/\" target=\"_blank\" rel=\"noopener\">Science category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/techxplore.com\/news\/2022-04-easier-robots-skills.html\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;An easier way to teach robots new skills&#8221; MIT researchers have developed a system that enables a robot to learn a new pick-and-place task based on only a handful of human examples. This could allow a human to reprogram a robot to grasp never-before-seen objects, presented in random poses, in about 15 minutes. Credit: Massachusetts&#8230;<\/p>\n","protected":false},"author":1,"featured_media":437173,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/scx2.b-cdn.net\/gfx\/news\/2022\/an-easier-way-to-teach.jpg","fifu_image_alt":"","footnotes":""},"categories":[16],"tags":[],"class_list":["post-437172","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sciencee"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/437172","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=437172"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/437172\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/437173"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=437172"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=437172"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=437172"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}