{"id":467518,"date":"2022-06-25T16:00:11","date_gmt":"2022-06-25T13:00:11","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/stop-debating-whether-ai-is-sentient-the-question-is-if-we-can-trust-it\/"},"modified":"2022-06-25T16:00:11","modified_gmt":"2022-06-25T13:00:11","slug":"stop-debating-whether-ai-is-sentient-the-question-is-if-we-can-trust-it","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/stop-debating-whether-ai-is-sentient-the-question-is-if-we-can-trust-it\/","title":{"rendered":"#Stop debating whether AI is \u2018sentient\u2019 \u2014 the question is if we can trust it"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a30655cf1ae9\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a30655cf1ae9\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/stop-debating-whether-ai-is-sentient-the-question-is-if-we-can-trust-it\/#%E2%80%9CStop_debating_whether_AI_is_%E2%80%98sentient_%E2%80%94_the_question_is_if_we_can_trust_it%E2%80%9D\" >&#8220;Stop debating whether AI is \u2018sentient\u2019 \u2014 the question is if we can trust it&#8221;<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/stop-debating-whether-ai-is-sentient-the-question-is-if-we-can-trust-it\/#Greetings_humanoids\" >Greetings, humanoids<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/stop-debating-whether-ai-is-sentient-the-question-is-if-we-can-trust-it\/#Why_large_language_models_dont_speak_our_language\" >Why large language models don\u2019t speak our language<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/stop-debating-whether-ai-is-sentient-the-question-is-if-we-can-trust-it\/#Sentience_vs_compatibility_and_trust\" >Sentience vs compatibility and trust<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/stop-debating-whether-ai-is-sentient-the-question-is-if-we-can-trust-it\/#The_problem-solver_perspective\" >The problem-solver perspective<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"%E2%80%9CStop_debating_whether_AI_is_%E2%80%98sentient_%E2%80%94_the_question_is_if_we_can_trust_it%E2%80%9D\"><\/span>&#8220;Stop debating whether AI is \u2018sentient\u2019 \u2014 the question is if we can trust it&#8221;<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<div id=\"article-main-content\">\n                            The past month has seen a frenzy of articles, interviews, and other types of <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">media<\/a> coverage about Blake Lemoine, a Google engineer who told <em><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.washingtonpost.com\/technology\/2022\/06\/11\/google-ai-lamda-blake-lemoine\/\">The Washington Post<\/a><\/em>\u00a0that LaMDA, a large language model created for conversations with users, is \u201csentient.\u201d<\/p>\n<p>After reading a dozen different takes on the topic, I have to say that the media has become (a bit) disillusioned with the hype surrounding current AI <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" data-internallinksmanager029f6b8e52c=\"4\" title=\"Technology\" target=\"_blank\" rel=\"noopener\">technology<\/a>. A lot of the articles discussed why deep neural networks are not \u201csentient\u201d or \u201cconscious.\u201d This is an improvement in comparison to a few years ago, when <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a> outlets were creating sensational stories about AI systems\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.forbes.com\/sites\/tonybradley\/2017\/07\/31\/facebook-ai-creates-its-own-language-in-creepy-preview-of-our-potential-future\/\">inventing their own language<\/a>, taking over every job, and accelerating toward\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/05\/13\/what-is-artificial-general-intelligence-agi\/\">artificial general intelligence<\/a>.<\/p>\n<p>But the fact that we\u2019re discussing sentience and consciousness again underlines an important point: We are at a point where our AI systems\u2014namely large language models\u2014are becoming increasingly convincing while still suffering from fundamental flaws that have been pointed out by scientists on different occasions. And I know that \u201cAI fooling humans\u201d has been discussed since the ELIZA chatbot in the 1960s, but today\u2019s LLMs are really at another level. If you don\u2019t know how language models work,\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/cajundiscordian.medium.com\/is-lamda-sentient-an-interview-ea64d916d917\">Blake Lemoine\u2019s conversations with LaMDA<\/a>\u00a0seem almost look surreal\u2014even if they had been cherry-picked and edited.<\/p>\n<div class=\"inarticle-wrapper neural channel-cta hs-embed-tnw\">\n<div id=\"hs-embed-tnw\" class=\"channel-cta-wrapper\">\n<div class=\"channel-cta-img\"><img decoding=\"async\" src=\"https:\/\/s3.amazonaws.com\/uploads.tnw\/uploads\/neural-newsletter_header-1.gif\"\/><\/div>\n<p><noscript><img decoding=\"async\" src=\"https:\/\/thenextweb.com\/news\/src=\" https:=\"\"\/><\/noscript><\/p>\n<div class=\"channel-cta-input\">\n<h2 class=\"channel-cta-title\"><span class=\"ez-toc-section\" id=\"Greetings_humanoids\"><\/span>Greetings, humanoids<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"channel-cta-tagline\">Subscribe to our newsletter now for a weekly recap of our favorite AI stories in your inbox.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>However, the point I want to make here is that \u201csentience\u201d and \u201cconsciousness\u201d is not the best discussion to have about LLMs and current AI technology. A more important discussion would be one about human compatibility and trust, especially since these technologies are being prepared to be integrated into everyday <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lications.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Why_large_language_models_dont_speak_our_language\"><\/span>Why large language models don\u2019t speak our language<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The workings of neural networks and large language models have been thoroughly discussed in the past week (I strongly recommend reading Melanie Mitchell\u2019s\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.msnbc.com\/opinion\/msnbc-opinion\/google-s-ai-impressive-it-s-not-sentient-here-s-n1296406\">interview with MSNBC<\/a>\u00a0for a balanced account of how LaMDA and other LLMs work). I would like to give a more zoomed-out view of the situation, starting with human language, with which LLMs are compared.<\/p>\n<p>For humans, language is a means to communicate the complicated and multi-dimensional activations happening in our brains. For example, when two brothers are talking to each other and one of them says \u201cmom,\u201d the word is associated with a lot of activations in different parts of the brain, including memories of her voice, face, feelings, and different experiences from the distant past to (possibly) recent days. In fact, there might be a huge difference between the kind of representations that the brothers hold in their brains, depending on the experiences that each has had. The word \u201cmom,\u201d however, provides a compressed and well-represented approximation that helps them agree on the same concept.<\/p>\n<p>When you use the word \u201cmom\u201d in a conversation with a stranger, the difference between the experiences and memories becomes even wider. But again, you manage to reach an agreement based on the shared concepts that you have in your minds.<\/p>\n<p>Think of language as a compression algorithm that helps transfer the enormous information in the brain to another person. The evolution of language is tied directly to experiences we\u2019ve had in the world, from physical interactions in our environment to social interactions with other fellow humans.<\/p>\n<p>Language is built on top of our shared experiences in the world. Children know about gravity, dimension, physical consistency of objects, and human and social concepts such as pain, sadness, fear, family, and friendship even before uttering their first word. Without those experiences, language has no meaning. This is why language usually omits commonsense knowledge and information that interlocutors share. On the other hand, the level of shared experience and memory will determine the depth of conversation you can have with another person.<\/p>\n<p>In contrast, large language models have no physical and social experience. They are trained on billions of words and learn to respond to prompts by predicting the next sequence of words. This is an approach that has yielded great results in the past few years, especially after the introduction of the\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/05\/02\/what-is-the-transformer\/\">transformer architecture<\/a>.<\/p>\n<p>How do transformers manage to make very convincing predictions? They turn text into \u201ctokens\u201d and \u201cembeddings,\u201d mathematical representations of words in a multi-dimensional space. They then process the embedding to add other dimensions such as the relations between the words in a sequence of text and their role in the sentence and paragraph. With enough examples, these embeddings can create good approximations of how words should appear in sequences. Transformers have become especially popular because they are scalable: Their accuracy improves as they become larger and are fed on more data, and they can be mostly trained through\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/02\/10\/unsupervised-learning-vs-supervised-learning\/\">unsupervised learning<\/a>.<\/p>\n<p>But the fundamental difference remains. Neural networks process language by turning them into embeddings. For humans, language\u00a0<em>is<\/em>\u00a0the embedding of thoughts, feelings, memory, physical experience, and many other things that we have yet to discover about the brain.<\/p>\n<p>This is why it is fair to say that despite their immense advances and impressive results, transformers, large language models, deep neural networks, etc. are still far from speaking our language.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Sentience_vs_compatibility_and_trust\"><\/span>Sentience vs compatibility and trust<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>A lot of the discussions today are about whether we should assign attributes such as sentience, consciousness, and personhood to AI. The problem with these discussions is that they are focused on concepts that are vaguely defined and mean different things to different people.<\/p>\n<p>For example, functionalists might argue that neural networks and large language models are conscious because they manifest (at least in part) the same kind of behavior that you would expect from a human, even though they are built on a different substrate. Others might argue that organic substance is a requirement for consciousness and conclude that neural networks will never be conscious. You can throw in arguments about qualia, the Chinese room experiment, the Turing test, etc., and the discussion can go on forever.<\/p>\n<p>However, a more practical question is, how \u201ccompatible\u201d are current neural networks with the human mind, and how far can we trust them with critical applications? And this is an important discussion to have because large language models are mostly developed by companies that seek to\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/05\/16\/opt-175b-large-language-models\/\">turn them into commercial applications<\/a>.<\/p>\n<p>For example, with enough training, you might be able to train a chimpanzee to ride a car. But would you put it behind a steering wheel on a road that pedestrians will be crossing? You wouldn\u2019t, because you know that however smart they are, chimpanzees don\u2019t think in the same way as humans and can\u2019t be given responsibility for tasks where human safety is concerned.<\/p>\n<p>Likewise, a parrot can be taught many phrases. But would you trust it to be your customer service agent? Probably not.<\/p>\n<p>Even when it comes to humans, some cognitive impairments disqualify people from taking on certain jobs and tasks that require human interactions or regard human safety. In many cases, these people can read, write, speak fluently, and remain consistent and logical in lengthy conversations. We don\u2019t question their sentience or consciousness or personhood. But we know that their decisions can become inconsistent and unpredictable due to their illness (see the case of\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.smithsonianmag.com\/history\/phineas-gage-neurosciences-most-famous-patient-11390067\/\">Phineas Gage<\/a>, for example).<\/p>\n<p>What matters is whether you can trust the person to think and decide as an average human would. In many cases, we trust people with tasks because we know that their sensory system, common-sense knowledge, feelings, goals, and rewards are mostly compatible with ours, even if they don\u2019t speak our language.<\/p>\n<p>What do we know about LaMDA? Well, for one thing, it doesn\u2019t sense the world as we do. Its \u201cknowledge\u201d of language isn\u2019t built on the same kind of experiences as ours. Its commonsense knowledge is built on an unstable foundation because there\u2019s no guarantee that large amounts of text will cover all the things we omit in language.<\/p>\n<p>Given this\u00a0<em>incompatibility<\/em>, how far can you\u00a0<em>trust<\/em>\u00a0LaMDA and other large language models, no matter how good they are at producing text output? A friendly and entertaining chatbot program might not be a bad idea as long as it doesn\u2019t steer the conversation into sensitive topics. Search engines are also a good application area for LLMs (Google has been\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/blog.google\/products\/search\/search-language-understanding-bert\/\">using BERT in search<\/a>\u00a0for a few years). But can you trust them with more sensitive tasks, such as an open-ended customer service chatbot or a banking advisor (even if they have been trained or finetuned on a ton of relevant conversation transcripts)?<\/p>\n<p>My thinking is that we\u2019ll need application-specific benchmarks to test the consistency of LLMs and their compatibility with human common sense in different areas. When it comes to real applications, there should always be clearly defined boundaries that determine where the conversation becomes off-limits for the LLM and should be handed to a human operator.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_problem-solver_perspective\"><\/span>The problem-solver perspective<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>A while back, I wrote an essay on\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/02\/14\/ai-humans-problem-solvers-problem-finders\/\">\u201cproblem finders\u201d and \u201cproblem solvers.\u201d<\/a>\u00a0Basically, what I said is that human intelligence is about finding the right problems and artificial intelligence (or the AI we have today) is about solving those problems in the most efficient manner.<\/p>\n<p>We have seen time and again that computers are able to find shortcuts for solving complicated problems without acquiring the cognitive abilities of humans. We\u2019ve seen it with checkers, chess, Go,\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/02\/07\/deepmind-alphacode-competitive-programming\/\">programming contests<\/a>, protein folding, and other well-defined problems.<\/p>\n<p>Natural language is in some ways different but also similar to all those other problems AI has solved. On the one hand, transformers and LLMs have shown that they can produce impressive results without going through the process of learning language like a normal human, which is to first explore the world and understand its basic rules and then acquire the language to interact with other people based on this common knowledge. On the other hand, they lack the human experience that comes with learning language. They can be useful for solving well-defined language-related problems. But we should not forget that their compatibility with human language processing is limited and thus we should be careful how far we trust them.<\/p>\n<p><em>This article was originally published by Ben Dickson on<span>\u00a0<\/span><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/\">TechTalks<\/a>, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech, and what we need to look out for. You can read the original article\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/06\/20\/lamda-large-language-models-sentient-ai\/\">here<\/a>.<\/em>\n                        <\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/stop-debating-whether-ai-is-sentient-trust\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;Stop debating whether AI is \u2018sentient\u2019 \u2014 the question is if we can trust it&#8221; The past month has seen a frenzy of articles, interviews, and other types of media coverage about Blake Lemoine, a Google engineer who told The Washington Post\u00a0that LaMDA, a large language model created for conversations with users, is \u201csentient.\u201d After&#8230;<\/p>\n","protected":false},"author":1,"featured_media":467519,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/06\/sentient.jpeg&signature=3a19e7aa65818472b054905e82d18749","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-467518","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/467518","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=467518"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/467518\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/467519"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=467518"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=467518"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=467518"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}