{"id":651587,"date":"2025-01-30T21:35:15","date_gmt":"2025-01-30T18:35:15","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster\/"},"modified":"2025-01-30T21:35:15","modified_gmt":"2025-01-30T18:35:15","slug":"what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster\/","title":{"rendered":"#What lawmakers can learn from South Korea&#8217;s AI hate-speech disaster"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a287a4245365\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a287a4245365\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster\/#Making_a_sexist_hateful_chatbot\" >Making a sexist, hateful chatbot<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster\/#A_pattern_of_digital_harassment\" >A pattern of digital harassment<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster\/#Beyond_privacy_The_human_cost\" >Beyond privacy: The human cost<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster\/#AI_ethics\" >AI ethics<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster\/#Canada_needs_strong_AI_policy\" >Canada needs strong AI policy<\/a><\/li><\/ul><\/nav><\/div>\n<div>\n<div class=\"article-gallery lightGallery\">\n<div data-thumb=\"https:\/\/scx1.b-cdn.net\/csz\/news\/tmb\/2025\/chatbot.jpg\" data-src=\"https:\/\/scx2.b-cdn.net\/gfx\/news\/hires\/2025\/chatbot.jpg\" data-sub-html=\"Credit: Pixabay\/CC0 Public Domain\">\n<figure class=\"article-img\">\n            <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/scx1.b-cdn.net\/csz\/news\/800a\/2025\/chatbot.jpg\" alt=\"chatbot\" title=\"Credit: Pixabay\/CC0 Public Domain\" width=\"800\" height=\"530\"\/><figcaption class=\"text-darken text-low-up text-truncate-js text-truncate mt-3\">\n                Credit: Pixabay\/CC0 Public Domain<br \/>\n            <\/figcaption><\/figure>\n<\/p><\/div>\n<\/div>\n<p>As artificial intelligence technologies develop at accelerated rates, the methods of governing companies and platforms continue to raise ethical and legal concerns.<\/p>\n<p>In Canada, many view proposed laws to regulate AI offerings as attacks on free speech and as overreaching government control on tech companies. This backlash has come from <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.nytimes.com\/2024\/04\/09\/world\/canada\/canada-hate-speech.html\" target=\"_blank\">free speech advocates<\/a>, <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/nationalpost.com\/opinion\/jordan-peterson-online-harms\" target=\"_blank\">right-wing figures<\/a> and <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.theatlantic.com\/ideas\/archive\/2024\/06\/canada-online-harms-act\/678605\/\" target=\"_blank\">libertarian<\/a> thought leaders.<\/p>\n<p>However, these critics should pay attention to a harrowing case from South Korea that offers important lessons about the risks of public-facing AI technologies and the critical need for user data protection.<\/p>\n<p>In late 2020, <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.theguardian.com\/world\/2021\/jan\/14\/time-to-properly-socialise-hate-speech-ai-chatbot-pulled-from-facebook\" target=\"_blank\">Iruda (or &#8220;Lee Luda&#8221;)<\/a>, an AI chatbot, quickly became a sensation in South Korea. AI chatbots are computer programs that simulate conversation with humans. In this case, the chatbot was designed as a 21-year-old female college student with a cheerful personality. Marketed as an exciting &#8220;AI friend,&#8221; Iruda attracted more than 750,000 users in under a month.<\/p>\n<p>But within weeks, Iruda became an ethics case study and a catalyst for addressing a lack of data governance in South Korea. She soon started to say troubling things and express hateful views. The situation was accelerated and exacerbated by the growing culture of digital sexism and sexual harassment online.<\/p>\n<p>                                                                                                        <!-- TechX - News - In-article --><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Making_a_sexist_hateful_chatbot\"><\/span>Making a sexist, hateful chatbot<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Scatter Lab, the tech startup that created Iruda, had already developed popular <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>s that analyzed emotions in text messages and offered dating advice. The company then used data from these apps to train Iruda&#8217;s abilities in intimate conversations. But it failed to fully disclose to users that their intimate messages would be used to train the chatbot.<\/p>\n<p>The problems began when users noticed Iruda repeating private conversations verbatim from the company&#8217;s dating advice apps. These responses included suspiciously real names, credit card information and home addresses, leading to an investigation.<\/p>\n<p>The chatbot also began expressing discriminatory and hateful views. Investigations by <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">media<\/a> outlets found this occurred after some users deliberately &#8220;trained&#8221; it with toxic language. Some users even created user guides on how to make Iruda a &#8220;sex slave&#8221; on popular online men&#8217;s forums. Consequently, Iruda began answering user prompts with <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/fpf.org\/blog\/south-korea-the-first-case-where-the-personal-information-protection-act-was-applied-to-an-ai-system\/\" target=\"_blank\">sexist, homophobic and sexualized hate speech<\/a>.<\/p>\n<p>This raised serious concerns about how AI and tech companies operate. The Iruda incident also raises concerns beyond policy and law for AI and tech companies. What happened with Iruda needs to be examined within a broader context of online sexual harassment in South Korea.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"A_pattern_of_digital_harassment\"><\/span>A pattern of digital harassment<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>South Korean <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.kci.go.kr\/kciportal\/ci\/sereArticleSearch\/ciSereArtiView.kci?sereArticleSearchBean.artiId=ART002923058\" target=\"_blank\">feminist scholars<\/a> have documented how digital platforms have become battlegrounds for gender-based conflicts, with co-ordinated campaigns targeting women who speak out on feminist issues. Social media amplifies these dynamics, creating what Korean American researcher Jiyeon Kim calls &#8220;<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/doi.org\/10.1080\/14680777.2017.1283343\" target=\"_blank\">networked misogyny<\/a>.&#8221;<\/p>\n<p>South Korea, home to the radical feminist 4B movement (which stands for four types of refusal against men: no dating, marriage, sex or children), provides an early example of the intensified gender-based conversations that are commonly seen online worldwide. As <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/benbellabooks.com\/shop\/flowers-of-fire\/\" target=\"_blank\">journalist Hawon Jung<\/a> points out, the corruption and abuse exposed by Iruda stemmed from existing social tensions and legal frameworks that refused to address online misogyny. Jung has written extensively on the decades-long struggle to prosecute hidden cameras and revenge porn.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Beyond_privacy_The_human_cost\"><\/span>Beyond privacy: The human cost<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Of course, Iruda was just one incident. The world has seen numerous other cases that demonstrate how seemingly harmless applications like AI chatbots can become vehicles for harassment and abuse without proper oversight.<\/p>\n<p>These include <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/spectrum.ieee.org\/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation\" target=\"_blank\">Microsoft&#8217;s Tay.ai in 2016<\/a>, which was manipulated by users to spout antisemitic and misogynistic tweets. More recently, <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.cnn.com\/2024\/10\/30\/tech\/teen-suicide-character-ai-lawsuit\/index.html\" target=\"_blank\">a custom chatbot on Character.AI was linked to a teen&#8217;s suicide<\/a>.<\/p>\n<p>Chatbots\u2014that appear as likable characters that feel increasingly human with rapid <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" data-internallinksmanager029f6b8e52c=\"4\" title=\"Technology\" target=\"_blank\" rel=\"noopener\">technology<\/a> advancements\u2014are uniquely equipped to extract deeply personal information from their users.<\/p>\n<p>These attractive and friendly AI figures exemplify what technology scholars Neda Atanasoski and Kalindi Vora describe as the logic of &#8220;<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.dukeupress.edu\/surrogate-humanity\" target=\"_blank\">surrogate humanity<\/a>&#8220;\u2014where AI systems are designed to stand in for human interaction but end up amplifying existing social inequalities.<\/p>\n<p>                                                                                                        <!-- TechX - News - In-article --><\/p>\n<h2><span class=\"ez-toc-section\" id=\"AI_ethics\"><\/span>AI ethics<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In South Korea, Iruda&#8217;s shutdown sparked a national conversation about AI ethics and data rights. The government responded by creating new AI guidelines and <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/en.yna.co.kr\/view\/AEN20210428009500315\" target=\"_blank\">fining Scatter Lab 103 million won<\/a> ($110,000 CAD).<\/p>\n<p>However, <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/doi.org\/10.1093\/idpl\/ipad006\" target=\"_blank\">Korean legal scholars Chea Yun Jung and Kyun Kyong Joo<\/a> note these measures primarily emphasized self-regulation within the tech industry rather than addressing deeper structural issues. It did not address how Iruda became a mechanism through which predatory male users disseminated misogynist beliefs and gender-based rage through deep learning technology.<\/p>\n<p>Ultimately, looking at AI regulation as a corporate issue is simply not enough. The way these chatbots extract private data and build relationships with human users means that feminist and community-based perspectives are essential for holding tech companies accountable.<\/p>\n<p>Since this incident, Scatter Lab has <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/biz.chosun.com\/en\/en-it\/2025\/01\/22\/SBVLWFY2U5BRPGGFCBHZZPXOQQ\/\" target=\"_blank\">been working with researchers to demonstrate the benefits of chatbots<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Canada_needs_strong_AI_policy\"><\/span>Canada needs strong AI policy<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In Canada, the proposed <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.justice.gc.ca\/eng\/csj-sjc\/pl\/charter-charte\/c27_1.html\" target=\"_blank\">Artificial Intelligence and Data Act<\/a> and <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.justice.gc.ca\/eng\/csj-sjc\/pl\/charter-charte\/c63.html\" target=\"_blank\">Online Harms Act<\/a> are still being shaped, and the boundaries of what constitutes a <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.whitecase.com\/insight-our-thinking\/ai-watch-global-regulatory-tracker-canada\" target=\"_blank\">&#8220;high-impact&#8221; AI system<\/a> remain undefined.<\/p>\n<p>The challenge for Canadian policymakers is to create frameworks that protect innovation while preventing systemic abuse by developers and malicious users. This means developing clear guidelines about data consent, implementing systems to prevent abuse, and establishing meaningful accountability measures.<\/p>\n<p>As AI becomes more integrated into our daily lives, these considerations will only become more critical. The Iruda case shows that when it comes to AI regulation, we need to think beyond technical specifications and consider the very real human implications of these technologies.<\/p>\n<div class=\"d-inline-block text-medium my-4\">\n                                                Provided by<br \/>\n                                                                                                    The Conversation<br \/>\n                                                    \t\t\t\t\t\t\t\t\t\t\t\t\t<a rel=\"nofollow\" target=\"_blank\" class=\"icon_open\" href=\"https:\/\/theconversation.com\" target=\"_blank\" rel=\"nofollow\"><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t<svg>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<use href=\"https:\/\/techx.b-cdn.net\/tmpl\/v2\/img\/svg\/sprite.svg#icon_open\" x=\"0\" y=\"0\"\/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/svg><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/a><\/p><\/div>\n<p class=\"article-main__note mt-4\">\n                                                This article is republished from <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/theconversation.com\" target=\"_blank\">The Conversation<\/a> under a Creative Commons license. Read the <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/theconversation.com\/from-chatbot-to-sexbot-what-lawmakers-can-learn-from-south-koreas-ai-hate-speech-disaster-247152\" target=\"_blank\">original article<\/a>.<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/counter.theconversation.com\/content\/247152\/count.gif?distributor=republish-lightbox-advanced\" alt=\"The Conversation\" width=\"1\" height=\"1\"\/>\n                                            <\/p>\n<p>                                        <!-- print only --><\/p>\n<div class=\"d-none d-print-block\">\n<p>\n                                                <strong>Citation<\/strong>:<br \/>\n                                                From chatbot to sexbot: What lawmakers can learn from South Korea&#8217;s AI hate-speech disaster (2025, January 30)<br \/>\n                                                retrieved 30 January 2025<br \/>\n                                                from https:\/\/techxplore.com\/<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>\/2025-01-chatbot-sexbot-lawmakers-south-korea.html\n                                            <\/p>\n<p>\n                                            This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no<br \/>\n                                            part may be reproduced without the written permission. The content is provided for information purposes only.\n                                            <\/p>\n<\/p><\/div>\n<\/p><\/div>\n<p><script id=\"facebook-jssdk\" async=\"\" src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more Like this articles, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/category\/sciencee\/\" target=\"_blank\" >Science category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/techxplore.com\/news\/2025-01-chatbot-sexbot-lawmakers-south-korea.html\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Credit: Pixabay\/CC0 Public Domain As artificial intelligence technologies develop at accelerated rates, the methods of governing companies and platforms continue to raise ethical and legal concerns. In Canada, many view proposed laws to regulate AI offerings as attacks on free speech and as overreaching government control on tech companies. This backlash has come from free&#8230;<\/p>\n","protected":false},"author":1,"featured_media":651588,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/scx2.b-cdn.net\/gfx\/news\/hires\/2025\/chatbot.jpg","fifu_image_alt":"","footnotes":""},"categories":[16],"tags":[],"class_list":["post-651587","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sciencee"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/651587","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=651587"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/651587\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/651588"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=651587"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=651587"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=651587"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}