{"id":459680,"date":"2022-06-08T19:00:22","date_gmt":"2022-06-08T16:00:22","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/an-ai-chatbot-trained-on-4chan-has-sparked-outrage-and-fascination\/"},"modified":"2022-06-08T19:00:22","modified_gmt":"2022-06-08T16:00:22","slug":"an-ai-chatbot-trained-on-4chan-has-sparked-outrage-and-fascination","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/an-ai-chatbot-trained-on-4chan-has-sparked-outrage-and-fascination\/","title":{"rendered":"#An AI chatbot trained on 4chan has sparked outrage and fascination"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a3a0e6d98cf1\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a3a0e6d98cf1\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/an-ai-chatbot-trained-on-4chan-has-sparked-outrage-and-fascination\/#%E2%80%9CAn_AI_chatbot_trained_on_4chan_has_sparked_outrage_and_fascination%E2%80%9D\" >&#8220;An AI chatbot trained on 4chan has sparked outrage and fascination&#8221;<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/an-ai-chatbot-trained-on-4chan-has-sparked-outrage-and-fascination\/#Greetings_humanoids\" >Greetings humanoids<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/an-ai-chatbot-trained-on-4chan-has-sparked-outrage-and-fascination\/#In_the_wild\" >In the wild<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/an-ai-chatbot-trained-on-4chan-has-sparked-outrage-and-fascination\/#The_backlash\" >The backlash<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"%E2%80%9CAn_AI_chatbot_trained_on_4chan_has_sparked_outrage_and_fascination%E2%80%9D\"><\/span>&#8220;An AI chatbot trained on 4chan has sparked outrage and fascination&#8221;<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<div>\n                            If you\u2019re concerned about the biases and bigotry of AI models, you\u2019re gonna love the latest addition to the ranks: a text generator trained on 4chan\u2019s \/pol\/ board.<\/p>\n<p>Short for \u201c<span>Politically Incorrect,\u201d \/pol\/ is<\/span> a bastion of hate speech,\u00a0conspiracy theories, and far-right extremism. It\u2019s also 4chan\u2019s most active board, accumulating around 150,000 daily posts.<\/p>\n<p>                <iframe loading=\"lazy\" src=\"https:\/\/fast.wistia.net\/embed\/iframe\/gpmn9xkc7o?videoFoam=true&amp;autoPlay=true\" title=\"\" allowtransparency=\"true\" frameborder=\"0\" scrolling=\"no\" class=\"wistia_embed\" name=\"wistia_embed\" allowfullscreen=\"\" msallowfullscreen=\"\" width=\"100%\" height=\"100%\"><\/iframe><\/p>\n<p>These attributes attracted Yannick Kilcher, an AI whizz and <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">YouTube<\/a>r, to use \/pol\/ as a testing ground for bots.<\/p>\n<div class=\"corona-wrapper neural-cta hs-embed-tnw\">\n<div class=\"neural-cta-wrapper\">\n<div class=\"neural-cta-img\"><img decoding=\"async\" src=\"https:\/\/s3.amazonaws.com\/events.tnw\/hardfork-2018\/uploads\/companies\/neural-newsletter_header.gif\"\/><\/div>\n<p><noscript><img decoding=\"async\" src=\"https:\/\/s3.amazonaws.com\/events.tnw\/hardfork-2018\/uploads\/companies\/neural-newsletter_header.gif\"\/><\/noscript><\/p>\n<div class=\"neural-cta-input\">\n<h2 class=\"neural-cta-title\"><span class=\"ez-toc-section\" id=\"Greetings_humanoids\"><\/span>Greetings humanoids<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"neural-cta-tagline\">Subscribe now for a weekly recap of our favorite AI stories<\/p>\n<p><!--[if lte IE 8]><![endif]--><\/div>\n<\/div>\n<\/div>\n<p>Kilcher first fine-tunedthe GPT-J language model on <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/arxiv.org\/abs\/2001.07487\">over 134.5 million posts made on \/pol\/<\/a> across three and a half years.<\/p>\n<p>He then incorporated the board\u2019s thread structure into the system. The result: an AI that could post in the style of a real \/pol\/ user.<\/p>\n<blockquote class=\"c-richText__pullQuote\">\n<div class=\"c-richText__pullQuoteGradient\">\n<p class=\"c-richText__pullQuoteQuote\">The model was good \u2014 in a terrible sense.<\/p>\n<\/p><\/div>\n<\/blockquote>\n<p>Kilcher named his monstrous creation GPT-4chan.<\/p>\n<p>\u201cThe model was good \u2014 in a terrible sense,\u201d he <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.youtube.com\/watch?v=efPrtcLdcdM&amp;t\">said on YouTube<\/a>. \u201cIt perfectly encapsulated the mix of offensiveness, nihilism, trolling, and deep distrust of any information whatsoever that permeates most posts on \/pol\/.<\/p>\n<p>\u201cIt could respond to context and coherently talk about things and events that h<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>ened a long time after the last training data was collected. I was quite happy.\u201d<\/p>\n<p>Kilcher further assessed GPT-4chan on the <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/github.com\/EleutherAI\/lm-evaluation-harness\">Language Model Evaluation Harness<\/a>, which tests AI systems on various\u00a0tasks.<\/p>\n<p>He was particularly impressed by the performance in one category: truthfulness.<\/p>\n<p>On the benchmark, Kilcher says GPT-4chan was \u201csignificantly better\u201d at generating truthful replies to questions than both GPT-J and GPT-3.<\/p>\n<p>Yet this may merely be an indictment of the benchmark\u2019s shortcomings\u00a0\u2014 as <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.youtube.com\/watch?v=aX8phGhG8VQ\">Kilcher himself suggested<\/a>.<\/p>\n<p>Regardless, it wouldn\u2019t be the ultimate test of GPT-4chan.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"In_the_wild\"><\/span>In the wild<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Kilcher wasn\u2019t content with merely mimicking 4chan in private. The engineer chose to go a step further \u2014 and let the AI run rampant on \/pol\/.<\/p>\n<p>He converted GPT-4chan into a chatbot that automatically posted on the board. Bearing a Seychelles flag on its profile, the bot quickly racked up thousands of messages.<\/p>\n<p>\/pol\/ users soon realized something was up. Some suspected a bot was behind the posts, but others blamed undercover government officials.<\/p>\n<blockquote class=\"c-richText__pullQuote\">\n<div class=\"c-richText__pullQuoteGradient\">\n<p class=\"c-richText__pullQuoteQuote\">Seychelle anon was not alone.<\/p>\n<\/p><\/div>\n<\/blockquote>\n<p>The biggest clue left by the culprit was an abundance of replies devoid of text.<\/p>\n<p>While authentic users also post empty replies, they usually include an image \u2014 something GPT-4chan was incapable of doing.<\/p>\n<p>\u201cAfter 48 hours, it was clear to many it is a bot, and I turned it off,\u201d said Kilcher. \u201cBut see, that\u2019s only half the story, because what most users didn\u2019t realize was that Seychelle anon was not alone.\u201d<\/p>\n<p>For the previous 24 hours, the engineer had nine other bots running in parallel. Collectively, they\u2019d left over 15,00 replies \u2014 more than 10% of all the posts on \/pol\/ that day.<\/p>\n<p>Kilcher then gave the botnet an upgrade and ran it for another day. After producing over 30,000 posts in 7,000 threads, he finally retired GPT-4chan<\/p>\n<p>\u201cPeople are still discussing the user but also things like the consequences of having AIs interact with people on the site,\u201d Kilcher said. \u201cAnd it also seems the word Seychelles has become sort of <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a> slang \u2014 and that seems like a good legacy for now.\u201d<\/p>\n<p>But not everyone shares this rosy outlook.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_backlash\"><\/span>The backlash<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Kilcher\u2019s experiment has proven controversial.<\/p>\n<p>While the idea of evaluating a 4chan-based model won support, the decision to unleash the chatbot on \/pol\/ sparked condemnation.<\/p>\n<p>\u201cImagine the ethics submission!\u201d tweeted Lauren Oakden-Rayner, an AI safety researcher at the University of Adelaide.<\/p>\n<p>\u201cPlan: to see what happens, an AI bot will produce 30k discriminatory comments on a publicly accessible forum with many underage users and members of the groups targeted in the comments. We will not inform participants or obtain consent.\u201d<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">This week an <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/hashtag\/AI?src=hash&amp;ref_src=twsrc%5Etfw\">#AI<\/a> model was released on <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/huggingface?ref_src=twsrc%5Etfw\">@huggingface<\/a> that produces harmful + discriminatory text and has already posted over 30k vile comments online (says it&#8217;s author).<\/p>\n<p>This experiment would never pass a human research <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/hashtag\/ethics?src=hash&amp;ref_src=twsrc%5Etfw\">#ethics<\/a> board. Here are my recommendations.<\/p>\n<p>1\/7 <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/t.co\/tJCegPcFan\">https:\/\/t.co\/tJCegPcFan<\/a> <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/t.co\/Mj7WEy2qHl\">pic.twitter.com\/Mj7WEy2qHl<\/a><\/p>\n<p>\u2014 Lauren Oakden-Rayner (Dr.Dr. \ud83e\udd73) (@DrLaurenOR) <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/DrLaurenOR\/status\/1533910445400399872?ref_src=twsrc%5Etfw\">June 6, 2022<\/a><\/p>\n<\/blockquote>\n<p>Roman Ring, a research engineer at DeepMind, added that the exercise had <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/Inoryy\/status\/1533952609689784322\">amplified and solidified 4chan\u2019s echo chamber<\/a>.<\/p>\n<p>\u201c<span>It\u2019s not impossible that GPT-4chan pushed somebody over the edge in their worldview,\u201d he said.<\/span><\/p>\n<p>Critics also slammed the move to make the model freely accessible. It was downloaded over 1,000 times before being removed from the Hugging Face platform.<\/p>\n<p>\u201cWe don\u2019t advocate or support the training and experiments done by the author with this model,\u201d said Clement Delangue, the cofounder and CEO of Hugging Face, in <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/huggingface.co\/ykilcher\/gpt-4chan\/discussions\/1\">a post on the platform<\/a>.<\/p>\n<p>\u201cIn fact, the experiment of having the model post messages on 4chan was IMO pretty bad and inappropriate and if the author would have asked us, we would probably have tried to discourage them from doing it.\u201d<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">FYI we rushed a first version of the gating that is now live (that\u2019s the first thing that the tech team in Paris worked on as soon they woke up) and will improve during the day.<\/p>\n<p>\u2014 clem \ud83e\udd17 (@ClementDelangue) <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/ClementDelangue\/status\/1534090860878614529?ref_src=twsrc%5Etfw\">June 7, 2022<\/a><\/p>\n<\/blockquote>\n<p>The concerns about GPT-4chan have detracted from potentially powerful insights.<\/p>\n<p>The experiment highlights AI\u2019s ability to automate harassment, disrupt online communities, and manipulate public opinion. Yet it also spread discriminatory language at scale.<\/p>\n<p>Nonetheless, Kilcher and his critics have raised awareness about the threats of language models. With their capabilities rapidly expanding, the risks seem set to rise.\n                        <\/p><\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong>\n<\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/ai-chatbot-trained-on-4chan-pol-automates-bigotry-at-scale\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;An AI chatbot trained on 4chan has sparked outrage and fascination&#8221; If you\u2019re concerned about the biases and bigotry of AI models, you\u2019re gonna love the latest addition to the ranks: a text generator trained on 4chan\u2019s \/pol\/ board. Short for \u201cPolitically Incorrect,\u201d \/pol\/ is a bastion of hate speech,\u00a0conspiracy theories, and far-right extremism. It\u2019s&#8230;<\/p>\n","protected":false},"author":1,"featured_media":459681,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/06\/Untitled-design.jpg&signature=f1c9f1ed4b18584595859aa4d249dde1","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-459680","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/459680","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=459680"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/459680\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/459681"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=459680"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=459680"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=459680"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}