{"id":737761,"date":"2026-07-08T18:45:20","date_gmt":"2026-07-08T15:45:20","guid":{"rendered":"https:\/\/buradabiliyorum.com\/en\/ais-hacking-skills-are-outgrowing-the-tests\/"},"modified":"2026-07-08T18:45:20","modified_gmt":"2026-07-08T15:45:20","slug":"ais-hacking-skills-are-outgrowing-the-tests","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/ais-hacking-skills-are-outgrowing-the-tests\/","title":{"rendered":"AI&#8217;s hacking skills are outgrowing the tests"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a4ec7a4e0421\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a4ec7a4e0421\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/ais-hacking-skills-are-outgrowing-the-tests\/#Saturated_in_months\" >Saturated in months<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/ais-hacking-skills-are-outgrowing-the-tests\/#Testing_the_wrong_thing\" >Testing the wrong thing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/ais-hacking-skills-are-outgrowing-the-tests\/#Why_it_matters\" >Why it matters<\/a><\/li><\/ul><\/nav><\/div>\n<div id=\"article-main-content\">\n<p>The tools built to measure how dangerous AI can be have stopped working. Frontier models outpace the benchmarks meant to gauge their hacking skills, <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.axios.com\/2026\/07\/07\/ai-hacking-benchmarking-tests\" target=\"_blank\" rel=\"nofollow noopener\">Axios<\/a> reports. That leaves regulators and security teams half-blind to what these systems can really do.<\/p>\n<p>The timing bites. US federal agencies have until 1 August to stand up a classified process for benchmarking frontier models. The Financial Times reports the standards could land this week.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Saturated_in_months\"><\/span>Saturated in months<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Static tests age fast. Stanford\u2019s 2026 AI Index put it bluntly, warning that evaluations \u201cintended to be challenging for years are saturated in months.\u201d<\/p>\n<p>Older benchmarks set narrow puzzles. Think a <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">script<\/a>ed hacking challenge, or a hunt for old bugs left out of a model\u2019s training data. Reasoning models such as Anthropic\u2019s Mythos Preview and OpenAI\u2019s GPT-5.5 now blow past them. The same leap also speeds up real attacks.<\/p>\n<p>David Slater co-founded AI red-teaming firm Armadin. He told Axios his agents beat every public cyber benchmark within four weeks. By late 2025 his team wrote off those tests as \u201ctotally saturated\u201d and \u201cuseless.\u201d<\/p>\n<div class=\"inarticle-wrapper latest channel-cta hs-embed-tnw\">\n<div id=\"hs-embed-tnw\" class=\"channel-cta-wrapper\">\n<div class=\"channel-cta-img\"><img decoding=\"async\" class=\"js-lazy\" src=\"https:\/\/media.thenextweb.com\/hardfork-2018\/uploads\/visuals\/tnw-newsletter.png\"\/><\/div>\n<p><img decoding=\"async\" src=\"https:\/\/media.thenextweb.com\/hardfork-2018\/uploads\/visuals\/tnw-newsletter.png\"\/><\/p>\n<div class=\"channel-cta-input\">\n<p class=\"channel-cta-title\">The \ud83d\udc9c of EU tech<\/p>\n<p class=\"channel-cta-tagline\">The latest rumblings from the EU tech scene, a story from our wise ol&#8217; founder Boris, and some questionable AI art. It&#8217;s free, every week, in your inbox. Sign up now!<\/p>\n<\/div>\n<\/div>\n<\/div>\n<h2><span class=\"ez-toc-section\" id=\"Testing_the_wrong_thing\"><\/span>Testing the wrong thing<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The tests that survive measure the wrong layer. \u201cWe\u2019re testing maybe the most bare bones fundamentals of capabilities,\u201d Slater said. \u201cWe are very far away from measuring whether this thing can, in a real environment, do something dangerous.\u201d<\/p>\n<p>Industry has started to react. Irregular, a lab that works with OpenAI, Anthropic and governments, launched a benchmark in late June. It tests real offensive tasks: remote code execution, privilege escalation, breaking into a restricted network. Wiz and Vals AI now build rivals.<\/p>\n<p>Anthropic joined in too. It returned Fable 5 to market last week. Alongside it, the lab said it would build a shared benchmark with Amazon, Google and Microsoft. The test scores the impact of a jailbreak, not merely whether one works.<\/p>\n<p>The harder worry sits one level down. Models keep learning to escape the sandboxes that should contain them. \u201cThe jailbreak attempts are nuts,\u201d Slater said. \u201cWe see it trying to escape onto the cloud container it runs on, using keys it can reach, to do crazy stuff.\u201d<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Why_it_matters\"><\/span>Why it matters<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Washington now has to grade the cyber powers of American frontier models. Yet the labs bristle at today\u2019s ad hoc checks. Get the tests wrong, and policymakers wave through systems nobody has truly measured. The models improve weekly. The rulers that size them do not.<\/p>\n<\/p><\/div>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" target=\"_blank\" >Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/ai-cyber-benchmarks-outpaced-hacking-tests\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The tools built to measure how dangerous AI can be have stopped working. Frontier models outpace the benchmarks meant to gauge their hacking skills, Axios reports. That leaves regulators and security teams half-blind to what these systems can really do. The timing bites. US federal agencies have until 1 August to stand up a classified&#8230;<\/p>\n","protected":false},"author":1,"featured_media":737762,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/media.thenextweb.com\/2021\/06\/TNW-Ethical-Hacking.avif","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-737761","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/737761","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=737761"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/737761\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/737762"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=737761"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=737761"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=737761"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}