{"id":441537,"date":"2022-05-04T00:25:01","date_gmt":"2022-05-03T21:25:01","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/ai-research-is-a-dumpster-fire-and-googles-holding-the-matches\/"},"modified":"2022-05-04T00:25:01","modified_gmt":"2022-05-03T21:25:01","slug":"ai-research-is-a-dumpster-fire-and-googles-holding-the-matches","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/ai-research-is-a-dumpster-fire-and-googles-holding-the-matches\/","title":{"rendered":"#AI research is a dumpster fire and Google\u2019s holding the matches"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a3cc6228f7cf\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a3cc6228f7cf\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/ai-research-is-a-dumpster-fire-and-googles-holding-the-matches\/#%E2%80%9CAI_research_is_a_dumpster_fire_and_Googles_holding_the_matches%E2%80%9D\" >&#8220;AI research is a dumpster fire and Google\u2019s holding the matches&#8221;<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/ai-research-is-a-dumpster-fire-and-googles-holding-the-matches\/#How_it_started\" >How it started<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/ai-research-is-a-dumpster-fire-and-googles-holding-the-matches\/#How_its_going\" >How it\u2019s going<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/ai-research-is-a-dumpster-fire-and-googles-holding-the-matches\/#The_mudslide_effect\" >The mudslide effect<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/ai-research-is-a-dumpster-fire-and-googles-holding-the-matches\/#The_solution\" >The solution<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"%E2%80%9CAI_research_is_a_dumpster_fire_and_Googles_holding_the_matches%E2%80%9D\"><\/span>&#8220;AI research is a dumpster fire and Google\u2019s holding the matches&#8221;<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<p><img decoding=\"async\" src=\"https:\/\/img-cdn.tnwcdn.com\/image?fit=796%2C417&amp;url=https%3A%2F%2Fcdn0.tnwcdn.com%2Fwp-content%2Fblogs.dir%2F1%2Ffiles%2F2022%2F05%2Fgoogledumpsterfire.jpg&amp;signature=72df8d8547d8205c5c42b0fcbc68f687\" \/><\/p>\n<div>\n                            The world of AI research is in shambles. From the academics prioritizing easy-to-monetize schemes over breaking novel ground, to the Silicon Valley elite using the threat of job loss to encourage corporate-friendly hypotheses, the system is a broken mess.<\/p>\n<p>And Google deserves a lion\u2019s share of the blame.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"How_it_started\"><\/span>How it started<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>There were <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.statista.com\/chart\/26862\/countries-and-regions-with-highest-number-of-ai-research-papers-submitted-per-year\/#:~:text=While%20the%20U.S.%20still%20leads%20the%20world%20with,an%20astronomical%20increase%20over%20the%20last%20two%20decades.\">approximately 85,000 research papers<\/a> published globally on the subject of AI\/ML in the year 2000. Fast-forward to 2021 and there were nearly twice as many published in the US alone.<\/p>\n<p>To say there\u2019s been an explosion in the field would be a massive understatement. This influx of researchers and new ideas has led to deep learning becoming one of the world\u2019s most important technologies.<\/p>\n<p>Between 2014 and 2021 big tech all but abandoned its \u201cweb first\u201d and \u201cmobile first\u201d principles to adopt \u201cAI first\u201d strategies.<\/p>\n<p>Now, in 2022, AI developers and researchers are in higher demand (and command more salary) than nearly any other jobs in tech outside of the C-suite.<\/p>\n<p>But this sort of unfettered growth also has a dark side. In the scramble to meet the market demand for deep learning-based products and services, the field\u2019s become as cutthroat and fickle as professional sports.<\/p>\n<p>In the past few years, we\u2019ve seen the the \u201c<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.technologyreview.com\/2018\/02\/21\/145289\/the-ganfather-the-man-whos-given-machines-the-gift-of-imagination\/\">GANfather<\/a>,\u201d Ian Goodfellow, jump ship from Google to <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">App<\/a>le, Timnit Gebru and others get fired from Google for dissenting opinions on the efficacy of research, and a virtual torrent of dubious AI papers manage to somehow clear peer-review.<\/p>\n<p>The flood of talent that arrived in the wake of the deep learning explosion also brought a mudslide of bad research, fraud, and corporate greed along with it.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"How_its_going\"><\/span>How it\u2019s going<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Google, more than any other company, bears responsibility for the modern AI paradigm. That means we need to give big G full marks for bringing natural language processing and image recognition to the masses.<\/p>\n<p>It also means we can credit Google with creating the researcher-eat-researcher environment that has some college students and their big-tech-partnered professors treating research papers as little more than bait for venture capitalists and corporate headhunters.<\/p>\n<p>At the top, Google\u2019s shown its willingness to hire the world\u2019s most talented researchers. And it\u2019s also demonstrated numerous times that it\u2019ll fire them in a heartbeat if they don\u2019t toe the company line.<\/p>\n<p>The company made headlines around the globe after firing Timnit Gebru, a researcher it\u2019d hired to help lead its AI ethics division, in December of 2020. Just a few months later it fired another member of the team, Margaret Mitchell.<\/p>\n<p>Google maintains that the researchers\u2019 work wasn\u2019t up to spec, but both women and numerous supporters claim the firings only occurred after they brought up ethical concerns over research the company\u2019s AI boss, Jeff Dean, had signed off on.<\/p>\n<p>It\u2019s now barely over a year later and history is repeating itself. Google <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.nytimes.com\/2022\/05\/02\/technology\/google-fires-ai-researchers.html\">fired another world-renowned AI researcher<\/a>, Satrajit Chatterjee, after he led a team of scientists in challenging another paper Dean had signed off.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_mudslide_effect\"><\/span>The mudslide effect<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>At the top, this means the competition for high-paying jobs is fierce. And the hunt for the next talented researcher or developer begins earlier than ever.<\/p>\n<p>Students working towards advanced degrees in the fields of machine learning and AI, who eventually want to work outside of academia, are expected to author or co-author research papers that demonstrate their talent.<\/p>\n<p>Unfortunately, the pipeline from academia to big tech or the VC-led startup world is littered with crappy papers written by students whose entire bent is writing algorithms that can be monetized.<\/p>\n<p>A quick Google Scholar search for \u201cnatural language processing,\u201d for example, shows nearly a million hits. Many of the papers listed have hundreds or thousands of citations.<\/p>\n<p>On the surface, this would indicate that NLP is a thriving subset of machine learning research that\u2019s gained attention from researchers around the globe.<\/p>\n<p>In fact, searches for \u201cartificial neural network,\u201d \u201ccomputer vision,\u201d and \u201creinforcement learning\u201d all brought up a similar glut of results.<\/p>\n<p>Unfortunately, a significant portion of AI and ML research is either intentionally fraudulent or full of bad <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/sciencee\/\" data-internallinksmanager029f6b8e52c=\"5\" title=\"Science\" target=\"_blank\" rel=\"noopener\">science<\/a>.<\/p>\n<p>What may have worked well in the past is quickly becoming a potentially outdated mode of communicating research.<\/p>\n<p>The Guardian\u2019s Stuart Richie recently penned <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.theguardian.com\/books\/2022\/apr\/11\/the-big-idea-should-we-get-rid-of-the-scientific-paper\">an article<\/a> wondering if we should do away with research papers altogether. According to them, science\u2019s problems are baked in pretty deep:<\/p>\n<blockquote><p>This system comes with big problems. Chief among them is the issue of publication bias: reviewers and editors are more likely to give a scientific paper a good write-up and publish it in their journal if it reports positive or exciting results. So scientists go to great lengths to hype up their studies, lean on their analyses so they produce \u201cbetter\u201d results, and sometimes even commit fraud in order to impress those all-important gatekeepers. This drastically distorts our view of what really went on.<\/p>\n<\/blockquote>\n<p>The problem is that the gatekeepers everyone is trying to impress tend to hold the keys to students\u2019 future employment and academics\u2019 admission into prestigious journals or conferences \u2014 researchers may fail to gain their approval at their own peril.<\/p>\n<p>And, even if a paper manages to make it through peer-review, there\u2019s no guarantee the people pushing things through aren\u2019t asleep at the switch.<\/p>\n<p>That\u2019s why Guillaume Cabanac, an associate professor of computer science at the University of Toulouse, created a project called the <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/dbrech.irit.fr\/pls\/apex\/f?p=9999:24:::NO\">Problematic Paper Screener<\/a> (PPS).<\/p>\n<p>The PPS uses automation to flag papers containing potentially problematic code, math, or verbiage. In the spirit of science and fairness, Cabanac ensures every paper that\u2019s flagged gets a manual review from humans. But the job\u2019s likely too big for a handful of humans to do in their spare time.<\/p>\n<p>According to a <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.spectrumnews.org\/news\/tainted-kids-other-odd-phrases-cropping-up-in-autism-studies\/\">report<\/a> from Spectrum <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">News<\/a>, there are a lot of problematic papers out there. And the majority have to do with machine learning and AI:<\/p>\n<blockquote><p>The screener deemed about 7,650 studies problematic, including more than 6,000 for having tortured phrases. Most papers containing tortured phrases seem to come from the fields of machine learning, artificial intelligence and engineering.<\/p>\n<\/blockquote>\n<p>Tortured phrases are terms that raise red flags to researchers because they attempt to describe a process or concept that\u2019s already well-established.<\/p>\n<p>For example, the use of terms such as \u201ccounterfeit neural\u201d or \u201cman-made neural\u201d could indicate the use of a thesaurus plug-in used by bad actors trying to get away with plagiarizing previous work.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_solution\"><\/span>The solution<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>While Google can\u2019t be blamed for everything untoward in the fields of machine learning and AI, it\u2019s played an outsized role in the devolution of peer-reviewed research.<\/p>\n<p>This isn\u2019t to say that Google doesn\u2019t also support and prop up the scientific community through open-source, financial aid, and research support. And we\u2019re certainly not trying to imply that everyone studying AI is just out to make a quick buck.<\/p>\n<p>But the system\u2019s set up to encourage the monetization of algorithms first, and to further the field second. In order for this to change, big tech and academia both need to commit to wholesale reform in how research is presented and reviewed.<\/p>\n<p>Currently, there is no widely recognized third-party verification authority for papers. The peer-review system is more like an honor code than a set of agreed-upon principles followed by institutions.<\/p>\n<p>However, there is precedence for the establishment and operation of an oversight committee with the reach, influence, and expertise to govern across academic boundaries: the <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.ncaa.com\/\">NCAA<\/a>.<\/p>\n<p>If we can unify a fair-competition system for thousands of amateur athletics programs, it\u2019s a safe bet we could form a governing body to establish guidelines for academic research and review.<\/p>\n<p>And, as far as Google goes, there\u2019s a better than nil chance that CEO Sundar Pichai\u2019s going to find himself summoned before congress again if the company continues to fire the researchers it hires to oversee its ethical AI programs.<\/p>\n<p>US capitalism means a business is typically free to hire and fire whoever they want, but shareholders and workers have rights too.<\/p>\n<p>Eventually, Google\u2019s going to have to commit to ethical research or it\u2019ll find itself unable to compete with the companies and organizations willing to.\n                        <\/p><\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong>\n<\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/ai-research-dumpster-fire-and-googles-holding-the-matches\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;AI research is a dumpster fire and Google\u2019s holding the matches&#8221; The world of AI research is in shambles. From the academics prioritizing easy-to-monetize schemes over breaking novel ground, to the Silicon Valley elite using the threat of job loss to encourage corporate-friendly hypotheses, the system is a broken mess. And Google deserves a lion\u2019s&#8230;<\/p>\n","protected":false},"author":1,"featured_media":441538,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/05\/googledumpsterfire.jpg&signature=206ea5a750dc650bd0c30fbd18f9f7ea","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-441537","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/441537","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=441537"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/441537\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/441538"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=441537"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=441537"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=441537"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}