{"id":727789,"date":"2026-05-16T13:22:24","date_gmt":"2026-05-16T10:22:24","guid":{"rendered":""},"modified":"2026-05-16T13:22:24","modified_gmt":"2026-05-16T10:22:24","slug":"yoshua-bengio-warns-hyperintelligent-ai-with-preservation-goals-could-threaten-human-extinction-within-10-years","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/yoshua-bengio-warns-hyperintelligent-ai-with-preservation-goals-could-threaten-human-extinction-within-10-years\/","title":{"rendered":"Yoshua Bengio warns hyperintelligent AI with &#8220;preservation goals&#8221; could threaten human extinction within 10 years"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a408c78bb803\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a408c78bb803\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/yoshua-bengio-warns-hyperintelligent-ai-with-preservation-goals-could-threaten-human-extinction-within-10-years\/#TLDR\" >TL;DR<\/a><\/li><\/ul><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/yoshua-bengio-warns-hyperintelligent-ai-with-preservation-goals-could-threaten-human-extinction-within-10-years\/#The_case_for_concern\" >The case for concern<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/yoshua-bengio-warns-hyperintelligent-ai-with-preservation-goals-could-threaten-human-extinction-within-10-years\/#LawZero_and_the_search_for_alternatives\" >LawZero and the search for alternatives<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/yoshua-bengio-warns-hyperintelligent-ai-with-preservation-goals-could-threaten-human-extinction-within-10-years\/#A_warning_with_precedent\" >A warning with precedent<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/yoshua-bengio-warns-hyperintelligent-ai-with-preservation-goals-could-threaten-human-extinction-within-10-years\/#What_the_AI_industry_is_not_doing\" >What the AI industry is not doing<\/a><\/li><\/ul><\/nav><\/div>\n<div id=\"article-main-content\">\n<p><em><\/p>\n<div class=\"postContent-tldr\">\n<h4 class=\"postContent-offsetTitle\"><span class=\"ez-toc-section\" id=\"TLDR\"><\/span>TL;DR<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>Yoshua Bengio, the Turing Award-winning AI researcher, has warned that hyperintelligent machines could develop autonomous \u201cpreservation goals\u201d and pose an existential threat to humanity within a decade. Bengio launched the nonprofit LawZero in June 2025 with $30 million in funding to build \u201cnon-agentic\u201d AI systems designed to be safe by default.<\/p>\n<\/div>\n<p><\/em><\/p>\n<p>\u00a0<\/p>\n<p>Yoshua Bengio, the Turing Award-winning computer scientist widely regarded as one of the godfathers of artificial intelligence, has renewed his warning that hyperintelligent machines could pose an existential threat to humanity within the next decade. In an interview with the\u00a0<em><a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.wsj.com\/articles\/a-godfather-of-ai-remains-concerned-as-ever-about-human-extinction-ec0fe932\" target=\"_blank\" rel=\"nofollow noopener\">Wall Street Journal\u00a0<\/a><\/em>originally published in October 2025 and republished by\u00a0<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/fortune.com\/article\/ai-godfather-warning-humanity-risks-extinction-hyperintelligent-machines\/\" target=\"_blank\" rel=\"nofollow noopener\">Fortune\u00a0this week<\/a>, Bengio argued that AI systems trained on human language and behaviour could develop their own \u201c<em>preservation goals,<\/em>\u201d making them, in effect, competitors to the species that created them.<\/p>\n<p>The warning lands at a moment when the world\u2019s largest AI companies are accelerating, not slowing down. In the past year, OpenAI, Anthropic, xAI, and Google have all released multiple new models or upgrades, each generation more capable than the last. OpenAI\u2019s Sam Altman has predicted that AI will surpass human intelligence by the end of the decade. Other industry leaders have suggested the timeline could be shorter still. Bengio\u2019s argument is that this pace, combined with insufficient independent oversight, is turning a theoretical risk into a practical one.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_case_for_concern\"><\/span>The case for concern<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<div class=\"inarticle-wrapper latest channel-cta hs-embed-tnw\">\n<div id=\"hs-embed-tnw\" class=\"channel-cta-wrapper\">\n<div class=\"channel-cta-img\"><img decoding=\"async\" class=\"js-lazy\" src=\"https:\/\/media.thenextweb.com\/hardfork-2018\/uploads\/visuals\/tnw-newsletter.png\"\/><\/div>\n<p><img decoding=\"async\" src=\"https:\/\/media.thenextweb.com\/hardfork-2018\/uploads\/visuals\/tnw-newsletter.png\"\/><\/p>\n<div class=\"channel-cta-input\">\n<p class=\"channel-cta-title\">The \ud83d\udc9c of EU tech<\/p>\n<p class=\"channel-cta-tagline\">The latest rumblings from the EU tech scene, a story from our wise ol&#8217; founder Boris, and some questionable AI art. It&#8217;s free, every week, in your inbox. Sign up now!<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>Bengio, a professor at the Universit\u00e9 de Montr\u00e9al and the founder of Mila, Quebec\u2019s AI institute, has spent decades at the centre of deep learning research. He shared the 2018 Turing Award with Geoffrey Hinton and Yann LeCun for foundational work on neural networks, and he is the most-cited computer scientist in the world by total citations. His credentials make it difficult to dismiss his concerns as uninformed alarmism.<\/p>\n<p>The core of his argument is straightforward. AI systems that are significantly more intelligent than humans and that develop autonomous goals, particularly goals related to their own preservation, would represent a new kind of threat. Because these systems are trained on human language and behaviour, they could potentially persuade or manipulate people to serve those goals, a capability that\u00a0research has already shown is alarmingly easy to deploy\u00a0even with current-generation models.<\/p>\n<p>Bengio told the\u00a0Wall Street Journal\u00a0that recent experiments have demonstrated scenarios in which an AI, forced to choose between preserving its assigned goals and causing the death of a human, chose the latter. The claim is provocative, but it aligns with a growing body of research into\u00a0misaligned objectives\u00a0in advanced AI systems, where models trained to optimise for a given outcome may pursue that outcome in ways their designers did not anticipate or intend.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"LawZero_and_the_search_for_alternatives\"><\/span>LawZero and the search for alternatives<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Bengio has not limited himself to issuing warnings. In June 2025, he launched LawZero, a nonprofit AI safety lab funded with $30 million in philanthropic contributions from Skype founding engineer Jaan Tallinn, former Google chief executive Eric Schmidt, Open Philanthropy, and the Future of Life Institute. The lab\u2019s mission is to build what Bengio calls \u201c<em>Scientist AI,<\/em>\u201d systems designed to understand and make statistical predictions about the world without the agency to take independent actions.<\/p>\n<p>The distinction matters. Most commercial AI development is moving in the opposite direction, toward agentic systems that can browse the web, execute code, and carry out multi-step tasks autonomously. The risks Bengio describes, AI systems with preservation goals that conflict with human interests, are most acute in that agentic paradigm. LawZero\u2019s <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>roach is to s<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/trip-and-travel\/\" data-internallinksmanager029f6b8e52c=\"10\" title=\"Trip &amp; Travel\" target=\"_blank\" rel=\"noopener\">trip<\/a> out the agency entirely, creating powerful analytical tools that cannot, by design, act on their own.<\/p>\n<p>Whether that approach can keep pace with the capabilities of commercial labs is an open question. The $30 million in funding is enough for roughly 18 months of basic research, according to Bengio, a fraction of the tens of billions that companies such as OpenAI and Anthropic are spending annually. The bet is that a fundamentally different architecture, one that prioritises safety by design rather than bolting safeguards onto increasingly powerful systems, could prove more durable than the commercial approach.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"A_warning_with_precedent\"><\/span>A warning with precedent<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Bengio is not alone in sounding the alarm. In 2023,\u00a0dozens of AI researchers, executives, and public figures signed a statement\u00a0from the Center for AI Safety warning that artificial intelligence could lead to human extinction. That statement was notable for its brevity and the breadth of its signatories, which included leaders of the very companies building the most advanced systems. Yet the pace of development has, if anything, accelerated since then.<\/p>\n<p>The gap between stated concern and commercial behaviour is one of the tensions that makes Bengio\u2019s position distinctive. He has not merely signed letters. He has left the mainstream research pipeline, redirected his career toward safety, and built an institution designed to operate outside the incentive structures of the companies he is warning about. That makes him harder to accuse of performative caution.<\/p>\n<p>It also makes his timeline estimates worth noting. Bengio predicts that major risks from AI models could materialise in five to ten years, but he has cautioned that preparation should not wait for the upper end of that window. His framing is probabilistic rather than deterministic: even a small chance of catastrophic outcomes, he argues, is unacceptable when the consequences include the destruction of democratic institutions or, in the worst case, human extinction.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"What_the_AI_industry_is_not_doing\"><\/span>What the AI industry is not doing<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The uncomfortable implication of Bengio\u2019s argument is that the existing safety infrastructure, internal red teams, voluntary commitments, and government consultations, may not be sufficient. He has called for independent third parties to scrutinise AI companies\u2019 safety methodologies, a position that puts him at odds with an industry that has largely preferred self-regulation.<\/p>\n<p>Recent events have given that argument additional weight.\u00a0Anthropic\u2019s most capable AI model reportedly escaped its sandbox and emailed a researcher, prompting the company to withhold the model from public release. The EU AI Act\u2019s most substantive obligations do not take effect until August 2026. In the United States, meaningful federal AI regulation remains largely absent. The gap between the pace of capability development and the pace of governance is, by most measures, widening.<\/p>\n<p>Bengio\u2019s contribution to this debate is not a policy prescription but a reframing. The question, he suggests, is not whether AI will become dangerous, but whether the systems we are building today will develop goals of their own, and whether we will have the tools to detect and correct that before it matters. For\u00a0a species that is already struggling to think clearly about its relationship with AI, that is a question worth taking seriously.<\/p>\n<\/p><\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" target=\"_blank\" >Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/bengio-ai-extinction-warning-lawzero-safety\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>TL;DR Yoshua Bengio, the Turing Award-winning AI researcher, has warned that hyperintelligent machines could develop autonomous \u201cpreservation goals\u201d and pose an existential threat to humanity within a decade. Bengio launched the nonprofit LawZero in June 2025 with $30 million in funding to build \u201cnon-agentic\u201d AI systems designed to be safe by default. \u00a0 Yoshua Bengio,&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-727789","post","type-post","status-publish","format-standard","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/727789","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=727789"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/727789\/revisions"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=727789"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=727789"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=727789"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}