{"id":139634,"date":"2020-12-22T16:00:25","date_gmt":"2020-12-22T13:00:25","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/algorithms-behaving-badly-2020-edition\/"},"modified":"2020-12-22T16:00:25","modified_gmt":"2020-12-22T13:00:25","slug":"algorithms-behaving-badly-2020-edition","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/algorithms-behaving-badly-2020-edition\/","title":{"rendered":"#Algorithms behaving badly: 2020 edition"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a35a3b9e69b9\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a35a3b9e69b9\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/algorithms-behaving-badly-2020-edition\/#The_racism_problem\" >The racism problem<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/algorithms-behaving-badly-2020-edition\/#Algorithms_that_make_renters_and_lower-income_peoples_lives_more_difficult\" >Algorithms that make renters\u2019 and lower-income people\u2019s lives more difficult<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/algorithms-behaving-badly-2020-edition\/#Policing_and_persecution\" >Policing and persecution<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/algorithms-behaving-badly-2020-edition\/#Workplace_surveillance\" >Workplace surveillance<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/algorithms-behaving-badly-2020-edition\/#Revenge_of_the_students\" >Revenge of the students<\/a><\/li><\/ul><\/nav><\/div>\n<p>&#8220;<strong>#Algorithms behaving badly: 2020 edition<\/strong>&#8221;<br \/>\n<img decoding=\"async\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/12\/1-copy-52-796x417.jpg\" \/><\/p>\n<div>\n<p>The perils of leaving important decisions to computer algorithms are pretty easily imagined (see, e.g., \u201cMinority Report,\u201d \u201cI, Robot,\u201d \u201cWar <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/game\/\" data-internallinksmanager029f6b8e52c=\"7\" title=\"Game\" target=\"_blank\" rel=\"noopener\">Game<\/a>s\u201d). In recent years, however, algorithms\u2019\u00a0 job de<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">script<\/a>ions have only grown.<\/p>\n<p>They are replacing humans when it comes to making tough decisions that companies and government agencies prefer to say are grounded in statistics and formulas rather than the jumbled calculations of a human brain. Some health insurers use algorithms to determine <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/themarkup.org\/ask-the-markup\/2020\/03\/03\/healthcare-algorithms-robot-medicine\">who gets medical care<\/a> and in what order of priority, instead of leaving that choice to doctors. Colleges use them to <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.usnews.com\/education\/best-colleges\/articles\/how-admissions-algorithms-could-affect-your-college-acceptance\">decide which applicants to admit<\/a>. And prototypes of self-driving cars use them to weigh how to <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/insights.techreview.com\/who-should-decide-how-algorithms-decide\/\">minimize harm during a traffic accident<\/a>.<\/p>\n<p>Some of that computational outsourcing springs from high hopes\u2014that computer algorithms would <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.nytimes.com\/2020\/09\/18\/business\/digital-mortgages.html\">take bias<\/a> out of the lending process, for instance, or <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.nature.com\/articles\/d41591-020-00027-9\">help researchers develop a safe COVID-19 vaccine<\/a> in record time.<\/p>\n<p>But it\u2019s been proven again and again that formulas inherit the biases of their creators. An algorithm is only as good as the data and principles that train it, and a person or people are largely in charge of what it\u2019s fed.<\/p>\n<p>Every year there are myriad new examples of algorithms that were either created for a cynical purpose, functioned to reinforce racism, or spectacularly failed to fix the problems they were built to solve. We know about most of them because whistleblowers, journalists, advocates, and academics took the time to dig into a black box of computational decision-making and found some dark materials.<\/p>\n<p>Here are some big ones from 2020.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_racism_problem\"><\/span><strong>The racism problem<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>A lot of problems with algorithmic decision-making come down to bias, but some instances are more explicit than others. The Markup <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/themarkup.org\/google-the-giant\/2020\/07\/23\/google-advertising-keywords-black-girls\">reported<\/a> that Google\u2019s ad portal connects the keywords \u201cBlack girls,\u201d \u201cAsian girls,\u201d and \u201cLatina girls\u201d (but not \u201cWhite girls\u201d) to porn. (Google blocked the automated suggestions after The Markup reached out to the company. In the meantime, Googles\u2019 search algorithm briefly sent our story to the first page of search results for the word \u201cporn.\u201d)<\/p>\n<p>Sometimes the consequences of such bias can be severe.<\/p>\n<p>Some medical algorithms are racially biased\u2014deliberately. A paper in the New England Journal of Medicine identified 13 examples of <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.nejm.org\/doi\/full\/10.1056\/NEJMms2004740\">race \u201ccorrections\u201d<\/a> integrated into tools used by doctors to determine who receives certain medical interventions, like heart surgery, antibiotics for urinary tract infections, and screenings for breast cancer. The tools assume patients of different races are at different risks for certain diseases\u2014assumptions not always well-grounded in <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/sciencee\/\" data-internallinksmanager029f6b8e52c=\"5\" title=\"Science\" target=\"_blank\" rel=\"noopener\">science<\/a>, according to the researchers. The result: <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.consumerreports.org\/medical-tests\/medical-algorithms-have-a-race-problem\/\">a Black man who needs a kidney transplant<\/a> was deemed not eligible, as Consumer Reports reported, among other disasters.<\/p>\n<p>A related issue emerged in a lawsuit against the National Football League: Black players allege it\u2019s much <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.nytimes.com\/2020\/08\/25\/sports\/football\/nfl-concussion-racial-bias.html\">harder to receive compensation for concussion-related dementia<\/a> because of the way the league evaluates neurocognitive function. Essentially, they say, the league assumes Black players inherently have lower cognitive function than White players and weighs their eligibility for payouts accordingly.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Algorithms_that_make_renters_and_lower-income_peoples_lives_more_difficult\"><\/span><strong>Algorithms that make renters\u2019 and lower-income people\u2019s lives more difficult<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>If you\u2019ve ever rented a home, and the chances are you have, as renting has skyrocketed since the 2008 financial crisis, a landlord has likely run you through a tenant screening service. Whatever results the background check algorithms spit out <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a>ly constitute the difference between getting to rent the home in question and getting denied\u2014and, The Markup found, those reports are <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/themarkup.org\/locked-out\/2020\/05\/28\/access-denied-faulty-automated-background-checks-freeze-out-renters\">often faulty<\/a>. The computer-generated reports confuse identities, misconstrue minor run-ins with law enforcement as criminal records, and misreport evictions. And what little oversight exists typically comes too late for the wrongfully denied.<\/p>\n<p>Similarly, MIT <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" data-internallinksmanager029f6b8e52c=\"4\" title=\"Technology\" target=\"_blank\" rel=\"noopener\">Technology<\/a> Review reported, lawyers who work with low-income people are finding themselves <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.technologyreview.com\/2020\/12\/04\/1013068\/algorithms-create-a-poverty-trap-lawyers-fight-back\/\">butting up against inscrutable, unaccountable algorithms<\/a>, created by private companies, that do things like decide which children enter foster care, allocate Medicaid services, and determine access to unemployment benefits.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Policing_and_persecution\"><\/span><strong>Policing and persecution<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>There\u2019s an enduring allure to the idea of predicting crimes before they happen, even as police department after police department has discovered <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/themarkup.org\/ask-the-markup\/2020\/08\/20\/does-predictive-police-technology-contribute-to-bias\">problems with data-driven<\/a> models.<\/p>\n<p>A case in point: the Pasco County Sheriff\u2019s Department, which The Tampa Bay Times found routinely <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/projects.tampabay.com\/projects\/2020\/investigations\/police-pasco-sheriff-targeted\/intelligence-led-policing\/\">monitored and harassed people it identified as potential criminals<\/a>. The department \u201csends deputies to find and interrogate anyone whose name appears\u201d on a list generated from \u201carrest histories, unspecified intelligence and arbitrary decisions by police analysts,\u201d the <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>paper reported. Deputies appeared at people\u2019s homes in the middle of the night to conduct searches and wrote tickets for minor things like missing mailbox numbers. Many of those targeted were minors. The sheriff\u2019s department, in response, said the newspaper was cherry-picking examples and conflating legitimate police tactics with harassment.<\/p>\n<p>Facial recognition software, another policing-related algorithmic tool, led to the faulty arrest and detention of a Detroit man for a crime he did not commit, The New York Times reported in an <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.nytimes.com\/2020\/06\/24\/technology\/facial-recognition-arrest.html\">article cataloging the technology\u2019s privacy, accuracy, and race problems<\/a>.<\/p>\n<p>And in a particularly chilling development, The Washington Post reported, the Chinese tech company Huawei has been testing tools that could scan faces in crowds for ethnic features and send \u201c<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.washingtonpost.com\/technology\/2020\/12\/08\/huawei-tested-ai-software-that-could-recognize-uighur-minorities-alert-police-report-says\/\">Uighur alarms<\/a>\u201d to authorities. The Chinese government has detained members of the Muslim minority group en masse in prison camps\u2014persecution that<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.nytimes.com\/2020\/09\/24\/world\/asia\/china-muslims-xinjiang-detention.html\"> appears to be expanding<\/a>. Huawei USA spokesperson Glenn Schloss told the Post the tool \u201cis simply a <span style=\"background-color: rgba(46, 146, 255, 0.2);\">test, and<\/span>\u00a0it has not seen real-world application.\u201d<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Workplace_surveillance\"><\/span><strong>Workplace surveillance<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Big employers are turning to algorithms to help monitor their workers. This year, Microsoft <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.theguardian.com\/technology\/2020\/dec\/02\/microsoft-apologises-productivity-score-critics-derided-workplace-surveillance\">apologized<\/a> after it enabled a Microsoft 365 feature that allowed managers to monitor and analyze their <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.theguardian.com\/technology\/2020\/nov\/26\/microsoft-productivity-score-feature-criticised-workplace-surveillance\">workers\u2019 \u201cproductivity.\u201d<\/a> The productivity score factored in things like an individual\u2019s participation in group chats and the number of emails sent.<\/p>\n<p>Meanwhile, Business Insider reported that Whole Foods uses heat maps, which weigh things like a number of employee complaints and the local unemployment rate, to predict<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.businessinsider.com\/whole-foods-tracks-unionization-risk-with-heat-map-2020-1\"> which stores might see unionization attempts<\/a>. Whole Foods is owned by Amazon, which has an <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.vice.com\/en\/article\/5dp3yn\/amazon-leaked-reports-expose-spying-warehouse-workers-labor-union-environmental-groups-social-movements\">elaborate apparatus<\/a> for monitoring worker behavior.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Revenge_of_the_students\"><\/span><strong>Revenge of the students<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Anyone searching for inspiration in the fight against an algorithm-dominated tomorrow might look to students in the United Kingdom, who <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/HUCKmagazine\/status\/1294985562106015750\">took to the streets<\/a> after the education system decided to use an algorithm to <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.wired.com\/story\/an-algorithm-determined-uk-students-grades-chaos-ensued\/\">give grades based on past performance during the pandemic<\/a>. Or <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.theverge.com\/2020\/9\/2\/21419012\/edgenuity-online-class-ai-grading-keyword-mashing-students-school-cheating-algorithm-glitch\">these kids<\/a>, who discovered their tests were being graded by an algorithm\u2014and then promptly figured out how to exploit it by essentially mashing up a bunch of keywords.<\/p>\n<p><em>This article was <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/themarkup.org\/2020-in-review\/2020\/12\/15\/algorithms-bias-racism-surveillance\">originally published on The Markup<\/a> and was republished under the <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/\">Creative Commons Attribution-NonCommercial-NoDerivatives<\/a><a rel=\"nofollow noopener\" target=\"_blank\"> license.<\/a><\/em><\/p>\n<p class=\"c-post-pubDate\">\n                                    Published December 22, 2020 \u2014 13:00 UTC\n                                <\/p>\n<\/p><\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><script data-src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js#xfbml=1&amp;appId=378011798897423&amp;version=v2.6\" id=\"socialSrcFacebook\" type=\"text\/template\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/neural\/2020\/12\/22\/algorithms-behaving-badly-2020-edition-syndication\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#Algorithms behaving badly: 2020 edition&#8221; The perils of leaving important decisions to computer algorithms are pretty easily imagined (see, e.g., \u201cMinority Report,\u201d \u201cI, Robot,\u201d \u201cWar Games\u201d). In recent years, however, algorithms\u2019\u00a0 job descriptions have only grown. They are replacing humans when it comes to making tough decisions that companies and government agencies prefer to say&#8230;<\/p>\n","protected":false},"author":1,"featured_media":139635,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/12\/1-copy-52.jpg&signature=e923d7faa36b6c2bc9516ed01a3a328e","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-139634","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/139634","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=139634"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/139634\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/139635"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=139634"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=139634"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=139634"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}