{"id":616383,"date":"2024-04-11T08:00:26","date_gmt":"2024-04-11T05:00:26","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/british-darpa-to-build-ai-gatekeepers-for-safety-guarantees\/"},"modified":"2024-04-11T08:00:26","modified_gmt":"2024-04-11T05:00:26","slug":"british-darpa-to-build-ai-gatekeepers-for-safety-guarantees","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/british-darpa-to-build-ai-gatekeepers-for-safety-guarantees\/","title":{"rendered":"#&#8217;British DARPA&#8217; to build AI gatekeepers for &#8216;safety guarantees&#8217;"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a35ffc62271c\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a35ffc62271c\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/british-darpa-to-build-ai-gatekeepers-for-safety-guarantees\/#The_gatekeeper_guarantee\" >The gatekeeper guarantee<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/british-darpa-to-build-ai-gatekeepers-for-safety-guarantees\/#The_British_DARPA\" >The British DARPA?<\/a><\/li><\/ul><\/nav><\/div>\n<div id=\"article-main-content\">\n                            A British R&amp;D unit today unveiled a futuristic vision of \u201cquantitative safety guarantees\u201d for AI.<\/p>\n<p>The Advanced Research and Invention Agency (ARIA) compares the guarantees to the high safety standards in nuclear power and passenger aviation. In the case of machine learning, the standards involve a probabilistic guarantee that no harm will result from a particular action.<\/p>\n<p>At the core of ARIA\u2019s plan is a \u201cgatekeeper\u201d AI.\u00a0 This digital sentinel will ensure that<span>\u00a0other AI agents only operate within the guardrails set for a specific <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lication.<\/span><\/p>\n<p>ARIA will direct \u00a359 million towards the scheme. By the programme\u2019s end , the agency intends to demonstrate a scalable proof-of-concept in one domain. Suggestions include <span>electricity grid balancing and supply chain management.<\/span><\/p>\n<p><span>If effective, the project could safeguard high-stakes AI applications, such as improving critical infrastructure or optimising clinical trials.\u00a0<\/span><\/p>\n<div class=\"inarticle-wrapper latest channel-cta hs-embed-tnw\">\n<div id=\"hs-embed-tnw\" class=\"channel-cta-wrapper\">\n<div class=\"channel-cta-img\"><img decoding=\"async\" class=\"js-lazy\" src=\"https:\/\/s3.amazonaws.com\/events.tnw\/hardfork-2018\/uploads\/visuals\/tnw-newsletter.png\"\/><\/div>\n<p><img decoding=\"async\" src=\"https:\/\/s3.amazonaws.com\/events.tnw\/hardfork-2018\/uploads\/visuals\/tnw-newsletter.png\"\/><\/p>\n<div class=\"channel-cta-input\">\n<p class=\"channel-cta-title\">The &lt;3 of EU tech<\/p>\n<p class=\"channel-cta-tagline\">The latest rumblings from the EU tech scene, a story from our wise ol&#8217; founder Boris, and some questionable AI art. It&#8217;s free, every week, in your inbox. Sign up now!<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>The program is the brainchild of David \u2018davidad\u2019 Dalrymple, who co-invented the popular cryptocurrency Filecoin.<\/p>\n<p>Dalrymple has also extensively researched technical AI safety, which sparked his interest in the gatekeeper approach. As the programme director of ARIA, he can now put his theory into practice.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_gatekeeper_guarantee\"><\/span>The gatekeeper guarantee<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>ARIA\u2019s gatekeepers <span>will draw on scientific world models and mathematical proofs. Dalrymple said the concept combines commercial and academic concepts.<\/span><\/p>\n<p><span>\u201cThe approaches being explored by big AI companies rely on finite samples and do not provide any guarantees about the behaviour of AI systems at deployment,\u201d he told TNW via email.<\/span><\/p>\n<p><span>\u201cMeanwhile, if we focus too heavily on academic approaches like formal logic, we run the risk of effectively trying to build AI capabilities from scratch. <\/span><\/p>\n<p><span>\u201cThe gatekeeper approach gives us the best of both worlds by tuning frontier capabilities as an engine to drive at speed, but along rails of mathematical reasoning.\u201d<\/span><\/p>\n<p>This fusion requires deep interdisciplinary <span>collaboration \u2014 which is where ARIA comes in.<\/span><\/p>\n<h2><span class=\"ez-toc-section\" id=\"The_British_DARPA\"><\/span>The British DARPA?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Established last year, ARIA funds \u201chigh-risk, high-reward\u201d research.\u00a0 The strategy has attracted comparisons to DARPA, the<span>\u00a0Pentagon\u2019s \u201cmad <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/sciencee\/\" data-internallinksmanager029f6b8e52c=\"5\" title=\"Science\" target=\"_blank\" rel=\"noopener\">science<\/a>\u201d unit.<\/span><\/p>\n<p><span>Dalrymple has drawn another parallel with DARPA. He compares ARIA\u2019s new project to <\/span><span><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.darpa.mil\/program\/high-assurance-cyber-military-systems\">DARPA\u2019s HACMS program<\/a>, which created an unhackable quadcopter. The project proved that formal verification can create bug-free software.<\/span><\/p>\n<p><span>\u201cVulnerabilities can be ruled out, but only with assumptions about the scope and speed of interventions that an attacker can make on the physical embodiment of a system,\u201d Dalrymple said.<\/span><\/p>\n<p>His plan builds on an approach endorsed by<span> Yoshua Bengio, a renowned computer scientist. A Turing Award winner, Bengio has also called for<\/span>\u00a0\u201c<span>quantitative safety guarantees.\u201d But he\u2019s been disappointed by the progress thus far.<\/span><\/p>\n<p><span>\u201cUnlike methods to build bridges, drugs or nuclear plants, current approaches to train frontier AI systems \u2014 the most capable AI systems currently in existence \u2014 do not allow us to obtain quantitative safety guarantees of any kind,\u201d Bengio wrote in <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/yoshuabengio.org\/2023\/05\/07\/ai-scientists-safe-and-useful-ai\/\">a blogpost<\/a> last year.<\/span><\/p>\n<p><span>Dalrymple has a chance to change that. That would also be a huge boost for ARIA, which has attracted <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/techmonitor.ai\/leadership\/innovation\/aria-uk-darpa-dominic-cummings\">scrutiny from politicians<\/a>.\u00a0<\/span><\/p>\n<p>Some lawmakers have questioned ARIA\u2019s budget. The body has won \u00a3800mn in funding over five years \u2014 a sizeable sum but <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.gov.uk\/government\/publications\/beis-research-and-development-rd-budget-allocations-2020-to-2021\/beis-research-and-development-budget-allocations-2020-to-2021\">a mere fraction<\/a> of other<span>\u00a0government research bodies. <\/span><\/p>\n<p><span>ARIA can also point to potential savings on the horizon. One programme it launched last month aims to train AI systems at 0.1% of the current cost.<\/span><\/p>\n<p><i><span style=\"font-weight: 400;\">One of the themes of this year\u2019s TNW Conference is Ren-AI-ssance: The AI-Powered Rebirth. If you want to go deeper into all things artificial intelligence, or simply experience the event (and say hi to our editorial team), we\u2019ve got something special for our loyal readers. Use the code TNWX<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">MEDIA<\/a> at checkout to get 30% off your <\/span><\/i><i><span style=\"font-weight: 400;\">business pass<\/span><\/i><i><span style=\"font-weight: 400;\">, <\/span><\/i><i><span style=\"font-weight: 400;\">investor pass<\/span><\/i><i><span style=\"font-weight: 400;\"> or startup packages (<\/span><\/i><i><span style=\"font-weight: 400;\">Bootstrap<\/span><\/i><i><span style=\"font-weight: 400;\"> &amp; <\/span><\/i><i><span style=\"font-weight: 400;\">Scaleup<\/span><\/i><i><span style=\"font-weight: 400;\">).<\/span><\/i>\n                        <\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong>\n<\/p><\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/british-darpa-aria-plans-ai-safety-gatekeepers\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A British R&amp;D unit today unveiled a futuristic vision of \u201cquantitative safety guarantees\u201d for AI. The Advanced Research and Invention Agency (ARIA) compares the guarantees to the high safety standards in nuclear power and passenger aviation. In the case of machine learning, the standards involve a probabilistic guarantee that no harm will result from a&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-616383","post","type-post","status-publish","format-standard","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/616383","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=616383"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/616383\/revisions"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=616383"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=616383"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=616383"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}