{"id":325966,"date":"2021-08-16T21:50:14","date_gmt":"2021-08-16T18:50:14","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/researchers-fooled-ai-into-ignoring-stop-signs-using-a-cheap-projector\/"},"modified":"2021-08-16T21:50:14","modified_gmt":"2021-08-16T18:50:14","slug":"researchers-fooled-ai-into-ignoring-stop-signs-using-a-cheap-projector","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/researchers-fooled-ai-into-ignoring-stop-signs-using-a-cheap-projector\/","title":{"rendered":"#Researchers fooled AI into ignoring stop signs using a cheap projector"},"content":{"rendered":"<p>&#8220;<strong>#Researchers fooled AI into ignoring stop signs using a cheap projector<\/strong>&#8221;<\/p>\n<div>A trio of researchers at Purdue today published pre-print research demonstrating a novel adversarial attack against computer vision systems that can make an AI see \u2013 or not see \u2013 whatever the attacker wants.<\/p>\n<p>It\u2019s something that could potentially affect self-driving vehicles, such as Tesla\u2019s, that rely on cameras to navigate and identify objects.<\/p>\n<p><strong>Up front:<\/strong> The researchers wanted to confront the problem of digital manipulation in the physical world. It\u2019s easy enough to hack a computer or fool an AI if you have physical access to it, but tricking a closed system is much harder.<\/p>\n<p>Per the team\u2019s <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/arxiv.org\/pdf\/2108.06247.pdf\">pre-print paper<\/a>:<\/p>\n<blockquote><p>Adversarial attacks and defenses today are predominantly driven by studies in the digital space where the attacker manipulates a digital image on a computer. The other form of attacks, which are the physical attacks, have been reported in the literature, but most of the existing ones are invasive in the sense that they need to touch the objects, for example, painting a stop sign, wearing a colored shirt, or 3D-printing a turtle.<\/p>\n<p>In this paper, we present a non-invasive attack using structured illumination. The new attack, called the OPtical ADversarial attack (OPAD), is based on a low-cost projector-camera system where we project calculated patterns to alter the <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>earance of the 3D objects.<\/p>\n<\/blockquote>\n<p><strong>Background:<\/strong> There are a lot of ways to try and trick an AI vision system. They use cameras to capture images and then run those images against a database to try and match them with similar images.<\/p>\n<p>If we wanted to stop an AI from scanning our face we could wear a Halloween mask. And if we wanted to stop an AI from seeing at all we could cover its cameras. But those solutions require a level of physical access that\u2019s often prohibitive for dastardly deed-doers.<\/p>\n<p>What the researchers have done here is come up with a novel way to attack a digital system in the physical world.<\/p>\n<p>They use a \u201clow-cost projector\u201d to shine an adversarial pattern \u2013 a specific arrangement of light, images, and shadows \u2014 that tricks the AI into misinterpreting what it\u2019s seeing.<\/p>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-1364110 size-full js-lazy\" alt=\"A screenshot from a pre-print paper demonstrating an adversarial attack on a basketball and a stop sign using a projector. \" width=\"2013\" height=\"1396\" sizes=\"auto, (max-width: 2013px) 100vw, 2013px\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack.jpg\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack.jpg 2013w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-280x194.jpg 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-389x270.jpg 389w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-195x135.jpg 195w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-796x552.jpg 796w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-1592x1104.jpg 1592w\"\/><figcaption>Credit: <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/arxiv.org\/pdf\/2108.06247.pdf\">Gnanasambandam, et al.<\/a><\/figcaption><figcaption><a rel=\"nofollow noopener\" target=\"_blank\" href=\"#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Feditorial.thenextweb.com%2Fneural%2F2021%2F08%2F16%2Fresearchers-tricked-ai-ignoring-stop-signs-using-cheap-projector%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: Screenshot from the Purdue pre-print paper.\" data-title=\"Share Screenshot from the Purdue pre-print paper. on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share Screenshot from the Purdue pre-print paper. on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>Screenshot from the Purdue pre-print paper.<\/figcaption><noscript><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-1364110 size-full\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack.jpg\" alt=\"A screenshot from a pre-print paper demonstrating an adversarial attack on a basketball and a stop sign using a projector. \" width=\"2013\" height=\"1396\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack.jpg 2013w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-280x194.jpg 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-389x270.jpg 389w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-195x135.jpg 195w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-796x552.jpg 796w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/adverserialstopattack-1592x1104.jpg 1592w\"\/><\/noscript><\/figure>\n<p>The main point of this kind of research is to discover potential dangers and then figure out how to stop them. To that end, the researchers say they\u2019ve learned a great deal about mitigating these kinds of attacks.<\/p>\n<p>Unfortunately, this is the kind of thing you either need infrastructure in place to deal with or you have to train your systems to defend against it ahead of time. Hypothetically-speaking, that means its possible this attack could become a live threat to camera-based AI systems\u00a0at any moment.<\/p>\n<p>That possibility exposes a major flaw in Tesla\u2018s vision-only system (most other manufacturers\u2019 autonomous vehicle systems use a combination of different sensor types).<\/p>\n<p>While it\u2019s arguable the company\u2019s vehicles are the most advanced on Earth, there\u2019s no infrastructure in place to mitigate the possibility of a projector attack on a moving vehicle\u2019s vision systems.<\/p>\n<p><strong>Quick take:<\/strong> Let\u2019s not get too hasty in our judgment of Tesla\u2018s decision-making when it comes to going vision-only. Having a single point of failure is obviously a bad thing, but the company\u2019s got <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/techcrunch.com\/2021\/08\/16\/u-s-safety-regulator-opens-investigation-into-tesla-autopilot-following-crashes-with-parked-emergency-vehicles\/\">bigger things to worry about right now<\/a> than this particular type of attack.<\/p>\n<p>First, the researchers didn\u2019t test the attack on driverless vehicles. It\u2019s possible that big tech and big auto both have this sort of thing figured out \u2013 we\u2019ll have to wait and see if the research crosses over.<\/p>\n<p>But that doesn\u2019t mean it\u2019s not something that could be adapted or developed to be a threat to any and all vision systems. A hack that can fool an AI system on a laptop into thinking a stop sign is a speed limit sign can, at least potentially, fool an AI system in a car into thinking the same thing.<\/p>\n<p>Secondly, the other potential uses for this kind of adversarial attack are pretty terrifying too.<\/p>\n<p>The research concludes with this statement:<\/p>\n<blockquote>\n<p>The success of OPAD demonstrates the possibility of using an optical system to alter faces or for long-range surveillance tasks. It would be interesting to see how these can be realized in the future.<\/p>\n<\/blockquote>\n<p>After which the researchers acknowledge that the work was partially funded by the US Army. It\u2019s not hard to imagine why the government\u00a0would have a vested interest in fooling facial recognition systems.<\/p>\n<p>You can read the whole paper here on <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/arxiv.org\/pdf\/2108.06247.pdf\">arXiv<\/a>.<\/p>\n<\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong>\n<\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/researchers-tricked-ai-ignoring-stop-signs-using-cheap-projector\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#Researchers fooled AI into ignoring stop signs using a cheap projector&#8221; A trio of researchers at Purdue today published pre-print research demonstrating a novel adversarial attack against computer vision systems that can make an AI see \u2013 or not see \u2013 whatever the attacker wants. It\u2019s something that could potentially affect self-driving vehicles, such as&#8230;<\/p>\n","protected":false},"author":1,"featured_media":325967,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2021\/08\/stopattack.jpg&signature=3ff95e62e62b4f1c42d18ca4e0b2b0d3","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-325966","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/325966","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=325966"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/325966\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/325967"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=325966"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=325966"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=325966"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}