{"id":585256,"date":"2023-08-01T19:39:27","date_gmt":"2023-08-01T16:39:27","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/deepminds-new-ai-controls-robotic-tasks-without-specific-training\/"},"modified":"2023-08-01T19:39:27","modified_gmt":"2023-08-01T16:39:27","slug":"deepminds-new-ai-controls-robotic-tasks-without-specific-training","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/deepminds-new-ai-controls-robotic-tasks-without-specific-training\/","title":{"rendered":"#DeepMind&#8217;s new AI controls robotic tasks without specific training"},"content":{"rendered":"<div id=\"article-main-content\">\n                            Google DeepMind has a new AI model that can direct robotic tasks it was never trained to perform.<\/p>\n<p>Named RT-2, the model learns from web and robotics data. It then turns this information into simple instructions for machines.<\/p>\n<p>In tests, the model was <span>asked to take actions never seen in the robotic data, such as placing oranges in a matching bowl. To follow these commands, the system had to translate knowledge from web-based data. <\/span>According to DeepMind, the model had a 62% success for these operations \u2014 double that of its predecessor, RT-1.<\/p>\n<p>\u201cJust like language models are trained on text from the web to learn <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a> ideas and concepts, RT-2 transfers knowledge from web data to inform robot behaviour,\u201d said <span>Vincent Vanhoucke, head of robotics at DeepMind. \u201c<\/span>In other words, RT-2 can speak robot.\u201d<\/p>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-1398791 size-full js-lazy\" alt=\"The model was tested on various emergent robotic skills that are not present in the robotics data and require knowledge transfer from web pre-training\" width=\"1492\" height=\"1176\" sizes=\"auto, (max-width: 1492px) 100vw, 1492px\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626.png\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626.png 1492w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626-266x210.png 266w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626-171x135.png 171w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626-343x270.png 343w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626-796x627.png 796w\"\/><figcaption><a rel=\"nofollow noopener\" target=\"_blank\" href=\"#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Feditorial.thenextweb.com%2Fdeep-tech%2F2023%2F08%2F01%2Fgoogle-deepmind-new-ai-model-rt2-controls-robots-untrained-tasks-using-web-data%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: RT-2 was tested on various robotic skills that weren\u2019t present in the robotics data. Credit: Google DeepMind\" data-title=\"Share RT-2 was tested on various robotic skills that weren\u2019t present in the robotics data. Credit: Google DeepMind on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share RT-2 was tested on various robotic skills that weren\u2019t present in the robotics data. Credit: Google DeepMind on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>RT-2 was tested on various robotic skills that weren\u2019t present in the robotics data. Credit: Google DeepMind<\/figcaption><noscript><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-1398791 size-full\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626.png\" alt=\"The model was tested on various emergent robotic skills that are not present in the robotics data and require knowledge transfer from web pre-training\" width=\"1492\" height=\"1176\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626.png 1492w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626-266x210.png 266w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626-171x135.png 171w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626-343x270.png 343w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb0c2c8236e724dc720_64c2873d3b6f7032a03db6c5_Fig203-e1690906834626-796x627.png 796w\"\/><\/noscript><\/figure>\n<p><span style=\"font-weight: 400;\">The tests showed RT-2 has impressive generalisation capabilities. It also has an improved semantic and visual understanding of robotic data that wasn\u2019t previously encountered.<\/span><\/p>\n<div class=\"inarticle-wrapper channel-cta\">\n<div class=\"ica-text\">\n<p class=\"ica-text__title\">Catch up on our conference talks<\/p>\n<p>Watch videos of our past talks for free with TNW All Access \u2192<\/p>\n<\/div>\n<\/div>\n<p><span style=\"font-weight: 400;\">N<\/span><span style=\"font-weight: 400;\">otably, the model can use rudimentary reasoning to follow new user commands. Impressively, it can even <span>perform multi-stage semantic reasoning. For instance, when instructed to pick an object that could be used as a hammer, RT-2 correctly identified a rock as the best option.<\/span><\/span><\/p>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-1398784 size-full js-lazy\" alt=\"Here we show an example of such reasoning and the robot\u2019s resulting behaviour\" width=\"1493\" height=\"1305\" sizes=\"auto, (max-width: 1493px) 100vw, 1493px\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983.png\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983.png 1493w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983-240x210.png 240w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983-154x135.png 154w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983-309x270.png 309w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983-796x696.png 796w\"\/><figcaption><a rel=\"nofollow noopener\" target=\"_blank\" href=\"#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Feditorial.thenextweb.com%2Fdeep-tech%2F2023%2F08%2F01%2Fgoogle-deepmind-new-ai-model-rt2-controls-robots-untrained-tasks-using-web-data%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: In one test, RT-2 figured out that a rock would be the best object to pick up as an improvised hammer. Credit: Google DeepMind\" data-title=\"Share In one test, RT-2 figured out that a rock would be the best object to pick up as an improvised hammer. Credit: Google DeepMind on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share In one test, RT-2 figured out that a rock would be the best object to pick up as an improvised hammer. Credit: Google DeepMind on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>In one test, RT-2 figured out that a rock would be the best object to pick up as an improvised hammer. Credit: Google DeepMind<\/figcaption><noscript><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-1398784 size-full\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983.png\" alt=\"Here we show an example of such reasoning and the robot\u2019s resulting behaviour\" width=\"1493\" height=\"1305\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983.png 1493w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983-240x210.png 240w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983-154x135.png 154w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983-309x270.png 309w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1d047fa442a8c3acb_64c256fd9390fd8ed049a2d3_Fig208-e1690906868983-796x696.png 796w\"\/><\/noscript><\/figure>\n<p>In another evaluation, the model was commanded to push a bottle of ketchup towards a blue cube.<\/p>\n<p>There were several items in the scene, but the only one in the training dataset was the cube. Nonetheless, RT-2 successfully pushed the ketchup towards the specified destination.<\/p>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-1398780 size-full js-lazy\" alt=\"RT-2 performs well on real robot Language Table tasks. None of the objects except the blue cube were present in the training data.\" width=\"1545\" height=\"619\" sizes=\"auto, (max-width: 1545px) 100vw, 1545px\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360.png\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360.png 1545w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-280x112.png 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-270x108.png 270w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-540x216.png 540w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-1536x615.png 1536w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-796x319.png 796w\"\/><figcaption><a rel=\"nofollow noopener\" target=\"_blank\" href=\"#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Feditorial.thenextweb.com%2Fdeep-tech%2F2023%2F08%2F01%2Fgoogle-deepmind-new-ai-model-rt2-controls-robots-untrained-tasks-using-web-data%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: RT-2 performed well in real-world tasks. Credit: Google DeepMind\" data-title=\"Share RT-2 performed well in real-world tasks. Credit: Google DeepMind on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share RT-2 performed well in real-world tasks. Credit: Google DeepMind on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>RT-2 performed well in real-world tasks. Credit: Google DeepMind<\/figcaption><noscript><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-1398780 size-full\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360.png\" alt=\"RT-2 performs well on real robot Language Table tasks. None of the objects except the blue cube were present in the training data.\" width=\"1545\" height=\"619\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360.png 1545w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-280x112.png 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-270x108.png 270w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-540x216.png 540w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-1536x615.png 1536w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/64c28bb1c73e93977fcc3ada_64c256a1aadd37257efbe4cd_Fig207-e1690906914360-796x319.png 796w\"\/><\/noscript><\/figure>\n<p><span style=\"font-weight: 400;\">DeepMind has heralded RT-2 as a breakthrough in artificial intelligence. The Londonlab says the model brings us closer to a future of helpful robots.<\/span><\/p>\n<p>\u201cNot only does RT-2 show how advances in AI are cascading rapidly into robotics, it shows enormous promise for more general-purpose robots,\u201d said Vanhoucke. \u201cWhile there is still a tremendous amount of work to be done to enable helpful robots in human-centered environments, RT-2 shows us an exciting future for robotics just within grasp.\u201d<\/p>\n<p>You can read the RT-2 study paper <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/robotics-transformer2.github.io\/assets\/rt2.pdf\">here<\/a>.<\/p>\n<p>\u00a0\n                        <\/p><\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong>\n<\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/google-deepmind-new-ai-model-rt2-controls-robots-untrained-tasks-using-web-data\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Google DeepMind has a new AI model that can direct robotic tasks it was never trained to perform. Named RT-2, the model learns from web and robotics data. It then turns this information into simple instructions for machines. In tests, the model was asked to take actions never seen in the robotic data, such as&#8230;<\/p>\n","protected":false},"author":1,"featured_media":585257,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/tnw-blurple?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2023\/08\/Untitled-design-2.jpg&signature=3cea29e2e2c17420da10e9330fa21a57","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-585256","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/585256","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=585256"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/585256\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/585257"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=585256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=585256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=585256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}