{"id":99572,"date":"2020-10-28T18:07:18","date_gmt":"2020-10-28T15:07:18","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/facebooks-chief-ai-scientist-says-gpt-3-is-not-a-very-good-qa-system\/"},"modified":"2020-10-28T18:07:18","modified_gmt":"2020-10-28T15:07:18","slug":"facebooks-chief-ai-scientist-says-gpt-3-is-not-a-very-good-qa-system","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/facebooks-chief-ai-scientist-says-gpt-3-is-not-a-very-good-qa-system\/","title":{"rendered":"#Facebook&#8217;s chief AI scientist says GPT-3 is &#8216;not a very good&#8217; Q&#038;A system"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a409b209a59a\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a409b209a59a\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/facebooks-chief-ai-scientist-says-gpt-3-is-not-a-very-good-qa-system\/#How_did_GPT-3_perform\" >How did GPT-3 perform?<\/a><\/li><\/ul><\/nav><\/div>\n<p>&#8220;<strong>#<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">Facebook<\/a>&#8217;s chief AI scientist says GPT-3 is &#8216;not a very good&#8217; Q&amp;A system<\/strong>&#8221;<\/p>\n<div>\n                                The GPT-3 language model has inspired both awe and fear since OpenAI unveiled the system in June. But one person who isn\u2019t overly impressed is Facebook\u2018s Yann LeCun.<\/p>\n<p>In <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/m.facebook.com\/yann.lecun\/posts\/10157253205637143?notif_id=1603803722095314&amp;notif_t=story_reshare&amp;ref=notif\">a Facebook post<\/a> published Tuesday, the social network\u2019s c<span>hief AI scientist said the text generator is \u201cnot very good\u201d as a question-answering or dialog system, and that other <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>roaches produce better results.<\/span><\/p>\n<p>\u201cIt\u2019s entertaining, and perhaps mildly useful as a creative help,\u201d LeCun wrote. \u201cBut trying to build intelligent machines by scaling up language models is like building high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.\u201d<\/p>\n<p>To support his claims, LeCun pointed to <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/www.nabla.com\/blog\/gpt-3\/?fbclid=IwAR0TJY0G7hkBWa4eUjcwkvF9G5be86VutR5XGlRSrDf-DTxIjP006pif7iE\">a new study of the model\u2019s<\/a>\u00a0performance in healthcare scenarios by <span>Nabla, a medtech firm cofounded by\u00a0two of his former colleagues at Facebook.<\/span><\/p>\n<p><em>[Read:\u00a0<span dir=\"auto\">What audience intelligence data tells us about the 2020 US presidential election]<\/span><\/em><\/p>\n<p>The researchers note that Open AI\u2019s GPT-3 guidelines put healthcare \u201cin the high stakes category because people rely on accurate medical information for life-or-death decisions, and mistakes here could result in serious harm.\u201d In addition, diagnosing medical or psychiatric conditions are unsupported uses of the model.<\/p>\n<p>Nonetheless, Nabla\u00a0tried it out\u00a0<span>on a range of healthcare use cases.<\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"How_did_GPT-3_perform\"><\/span>How did GPT-3 perform?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The researchers found that GPT-3 seemed helpful in finding information in long documents and in basic admin tasks such as appointment booking. But it lacked the memory, logic, and understanding of time for many more specific questions.<\/p>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-1325906 lazy\" alt=\"\" width=\"1250\" height=\"1210\" sizes=\"auto, (max-width: 1250px) 100vw, 1250px\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.16.15.png\" data-lazy=\"true\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.16.15.png 1250w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.16.15-217x210.png 217w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.16.15-279x270.png 279w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.16.15-139x135.png 139w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.16.15-796x771.png 796w\"\/><figcaption>Credit: Nabla<\/figcaption><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/10\/28\/facebooks-yann-lecun-says-gpt-3-is-not-very-good-as-a-qa-or-dialog-system\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F10%2F28%2Ffacebooks-yann-lecun-says-gpt-3-is-not-very-good-as-a-qa-or-dialog-system%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: GPT-3 overlooked the patient\u2019s 6PM time deadline.\" data-title=\"Share GPT-3 overlooked the patient\u2019s 6PM time deadline. on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share GPT-3 overlooked the patient\u2019s 6PM time deadline. on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>GPT-3 overlooked the patient\u2019s 6PM time deadline.<\/figcaption><\/figure>\n<p>Nabla also found that GPT-3 was an unreliable Q&amp;A support tool for doctors, dangerously oversimplified medical documentation analysis, and struggled to associate causes with consequences.<\/p>\n<p>The model also made some basic errors in diagnosis and provided some reckless mental health advice.<\/p>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-1325907 lazy\" alt=\"\" width=\"1242\" height=\"506\" sizes=\"auto, (max-width: 1242px) 100vw, 1242px\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.20.22.png\" data-lazy=\"true\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.20.22.png 1242w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.20.22-280x114.png 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.20.22-540x220.png 540w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.20.22-270x110.png 270w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Screenshot-2020-10-28-at-13.20.22-796x324.png 796w\"\/><figcaption>Credit: Nabla<\/figcaption><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/10\/28\/facebooks-yann-lecun-says-gpt-3-is-not-very-good-as-a-qa-or-dialog-system\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F10%2F28%2Ffacebooks-yann-lecun-says-gpt-3-is-not-very-good-as-a-qa-or-dialog-system%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: GPT-3\u2019s suicide recommendation shows the model\u2019s need for constraints.\" data-title=\"Share GPT-3\u2019s suicide recommendation shows the model\u2019s need for constraints. on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share GPT-3\u2019s suicide recommendation shows the model\u2019s need for constraints. on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>GPT-3\u2019s suicide recommendation shows the model\u2019s need for constraints.<\/figcaption><\/figure>\n<p>The researchers do see some potential for using language models in medical settings. But they conclude that GPT-3 is \u201cnowhere near\u201d ready to provide significant help in the sector:<\/p>\n<blockquote><p><span>Because of the way it was trained, it lacks the scientific and medical expertise that would make it useful for medical documentation, diagnosis support, treatment recommendation or any medical Q&amp;A. Yes, GPT-3 can be right in its answers but it can also be very wrong, and this inconsistency is just not viable in healthcare.<\/span><\/p>\n<\/blockquote>\n<p>Their findings won\u2019t shock OpenAI, given the firm\u2019s warnings against using GPT-3 in healthcare. But they do show that many expectations for the model are wildly unrealistic.<\/p>\n<p class=\"c-post-pubDate\">\n                                    Published October 28, 2020 \u2014 15:07 UTC<\/p><\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><script data-src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js#xfbml=1&amp;appId=378011798897423&amp;version=v2.6\" id=\"socialSrcFacebook\" type=\"text\/template\"><\/script><\/p>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener noreferrer\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/neural\/2020\/10\/28\/facebooks-yann-lecun-says-gpt-3-is-not-very-good-as-a-qa-or-dialog-system\/\" target=\"_blank\" rel=\"noopener noreferrer\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#Facebook&#8217;s chief AI scientist says GPT-3 is &#8216;not a very good&#8217; Q&amp;A system&#8221; The GPT-3 language model has inspired both awe and fear since OpenAI unveiled the system in June. But one person who isn\u2019t overly impressed is Facebook\u2018s Yann LeCun. In a Facebook post published Tuesday, the social network\u2019s chief AI scientist said the&#8230;<\/p>\n","protected":false},"author":1,"featured_media":99573,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/10\/Untitled-design-2020-10-28T144028.713.png&signature=1fb6a71113d24f67e1bc54deaf4a7943","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-99572","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/99572","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=99572"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/99572\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/99573"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=99572"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=99572"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=99572"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}