{"id":673340,"date":"2025-06-03T22:55:14","date_gmt":"2025-06-03T19:55:14","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/new-tool-boosts-model-transparency\/"},"modified":"2025-06-03T22:55:14","modified_gmt":"2025-06-03T19:55:14","slug":"new-tool-boosts-model-transparency","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/new-tool-boosts-model-transparency\/","title":{"rendered":"New tool boosts model transparency"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a2db7d8b949c\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a2db7d8b949c\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/new-tool-boosts-model-transparency\/#Helping_models_know_what_they_dont_know\" >Helping models know what they don&#8217;t know<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/new-tool-boosts-model-transparency\/#Research_for_impact\" >Research for impact<\/a><\/li><\/ul><\/nav><\/div>\n<div>\n<div class=\"article-gallery lightGallery\">\n<div data-thumb=\"https:\/\/scx1.b-cdn.net\/csz\/news\/tmb\/2024\/ai-and-data-1.jpg\" data-src=\"https:\/\/scx2.b-cdn.net\/gfx\/news\/hires\/2024\/ai-and-data-1.jpg\" data-sub-html=\"Credit: Pixabay\/CC0 Public Domain\">\n<figure class=\"article-img\">\n            <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/scx1.b-cdn.net\/csz\/news\/800a\/2024\/ai-and-data-1.jpg\" alt=\"ai and data\" title=\"Credit: Pixabay\/CC0 Public Domain\" width=\"800\" height=\"448\"\/><figcaption class=\"text-darken text-low-up text-truncate-js text-truncate mt-3\">\n                Credit: Pixabay\/CC0 Public Domain<br \/>\n            <\/figcaption><\/figure>\n<\/p><\/div>\n<\/div>\n<p>Artificial intelligence systems like ChatGPT provide plausible-sounding answers to any question you might ask. But they don&#8217;t always reveal the gaps in their knowledge or areas where they&#8217;re uncertain. That problem can have huge consequences as AI systems are increasingly used to do things like develop drugs, synthesize information, and drive autonomous cars.<\/p>\n<p>Now, the MIT spinout Themis AI is helping quantify model uncertainty and correct outputs before they cause bigger problems. The company&#8217;s Capsa platform can work with any machine-learning model to detect and correct unreliable outputs in seconds. It works by modifying AI models to enable them to detect patterns in their data processing that indicate ambiguity, incompleteness, or bias.<\/p>\n<p>&#8220;The idea is to take a model, wrap it in Capsa, identify the uncertainties and failure modes of the model, and then enhance the model,&#8221; says Themis AI co-founder and MIT Professor Daniela Rus, who is also the director of the MIT Computer <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/sciencee\/\" data-internallinksmanager029f6b8e52c=\"5\" title=\"Science\" target=\"_blank\" rel=\"noopener\">Science<\/a> and Artificial Intelligence Laboratory (CSAIL). &#8220;We&#8217;re excited about offering a solution that can improve models and offer guarantees that the model is working correctly.&#8221;<\/p>\n<p>Rus founded Themis AI in 2021 with Alexander Amini &#8217;17, SM &#8217;18, Ph.D. &#8217;22 and Elaheh Ahmadi &#8217;20, MEng &#8217;21, two former research affiliates in her lab. Since then, they&#8217;ve helped telecom companies with network planning and automation, helped oil and gas companies use AI to understand seismic imagery, and published papers on developing more reliable and trustworthy chatbots.<\/p>\n<p>&#8220;We want to enable AI in the highest-stakes <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lications of every industry,&#8221; Amini says. &#8220;We&#8217;ve all seen examples of AI hallucinating or making mistakes. As AI is deployed more broadly, those mistakes could lead to devastating consequences. Our software can make these systems more transparent.&#8221;<\/p>\n<p>                                                                                                        <!-- TechX - News - In-article --><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Helping_models_know_what_they_dont_know\"><\/span>Helping models know what they don&#8217;t know<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Rus&#8217;s lab has been researching model uncertainty for years. In 2018, she received funding from Toyota to study the reliability of a machine-learning-based autonomous driving solution.<\/p>\n<p>&#8220;That is a safety-critical context where understanding model reliability is very important,&#8221; Rus says.<\/p>\n<p>In separate <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3306618.3314243\" target=\"_blank\">work<\/a>published in 2019, Rus, Amini, and their collaborators built an algorithm that could detect racial and gender bias in facial recognition systems and automatically reweight the model&#8217;s training data, showing it eliminated bias. The algorithm worked by identifying the unrepresentative parts of the underlying training data and generating new, similar data samples to rebalance it.<\/p>\n<p>In 2021, the eventual co-founders showed a <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/pubs.acs.org\/doi\/full\/10.1021\/acscentsci.1c00546\" target=\"_blank\">similar approach<\/a> could be used to help pharmaceutical companies use AI models to predict the properties of drug candidates. They founded Themis AI later that year.<\/p>\n<p>&#8220;Guiding drug discovery could potentially save a lot of money,&#8221; Rus says. &#8220;That was the use case that made us realize how powerful this tool could be.&#8221;<\/p>\n<p>Today Themis is working with companies in a wide variety of industries, and many of those companies are building large language models. By using Capsa, the models are able to quantify their own uncertainty for each output.<\/p>\n<p>&#8220;Many companies are interested in using LLMs that are based on their data, but they&#8217;re concerned about reliability,&#8221; observes Stewart Jamieson SM &#8217;20, Ph.D. &#8217;24, Themis AI&#8217;s head of <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" data-internallinksmanager029f6b8e52c=\"4\" title=\"Technology\" target=\"_blank\" rel=\"noopener\">technology<\/a>. &#8220;We help LLMs self-report their confidence and uncertainty, which enables more reliable question answering and flagging unreliable outputs.&#8221;<\/p>\n<p>Themis AI is also in discussions with semiconductor companies building AI solutions on their chips that can work outside of cloud environments.<\/p>\n<p>&#8220;Normally these smaller models that work on phones or embedded systems aren&#8217;t very accurate compared to what you could run on a server, but we can get the best of both worlds: low latency, efficient edge computing without sacrificing quality,&#8221; Jamieson explains. &#8220;We see a future where edge devices do most of the work, but whenever they&#8217;re unsure of their output, they can forward those tasks to a central server.&#8221;<\/p>\n<p>Pharmaceutical companies can also use Capsa to improve AI models being used to identify drug candidates and predict their performance in clinical trials.<\/p>\n<p>&#8220;The predictions and outputs of these models are very complex and hard to interpret\u2014experts spend a lot of time and effort trying to make sense of them,&#8221; Amini remarks. &#8220;Capsa can give insights right out of the gate to understand if the predictions are backed by evidence in the training set or are just speculation without a lot of grounding. That can accelerate the identification of the strongest predictions, and we think that has a huge potential for societal good.&#8221;<\/p>\n<p>                                                                                                        <!-- TechX - News - In-article --><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Research_for_impact\"><\/span>Research for impact<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Themis AI&#8217;s team believes the company is well-positioned to improve the cutting edge of constantly evolving AI technology. For instance, the company is exploring Capsa&#8217;s ability to improve accuracy in an AI technique known as chain-of-thought reasoning, in which LLMs explain the steps they take to get to an answer.<\/p>\n<p>&#8220;We&#8217;ve seen signs Capsa could help guide those reasoning processes to identify the highest-confidence chains of reasoning,&#8221; Amini says. &#8220;We think that has huge implications in terms of improving the LLM experience, reducing latencies, and reducing computation requirements. It&#8217;s an extremely high-impact opportunity for us.&#8221;<\/p>\n<p>For Rus, who has co-founded several companies since coming to MIT, Themis AI is an opportunity to ensure her MIT research has impact.<\/p>\n<p>&#8220;My students and I have become increasingly passionate about going the extra step to make our work relevant for the world,&#8221; Rus says. &#8220;AI has tremendous potential to transform industries, but AI also raises concerns. What excites me is the opportunity to help develop technical solutions that address these challenges and also build trust and understanding between people and the technologies that are becoming part of their daily lives.&#8221;<\/p>\n<div class=\"d-inline-block text-medium my-4\">\n                                                Provided by<br \/>\n                                                                                                    Massachusetts Institute of Technology<br \/>\n                                                    \t\t\t\t\t\t\t\t\t\t\t\t\t<a rel=\"nofollow\" target=\"_blank\" class=\"icon_open\" href=\"http:\/\/web.mit.edu\/\" target=\"_blank\" rel=\"nofollow\"><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t<svg>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<use href=\"https:\/\/techx.b-cdn.net\/tmpl\/v2\/img\/svg\/sprite.svg#icon_open\" x=\"0\" y=\"0\"\/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/svg><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<\/a><\/p><\/div>\n<p>                                        <!-- print only --><\/p>\n<div class=\"d-none d-print-block\">\n<p>\n                                                <strong>Citation<\/strong>:<br \/>\n                                                AI learns to admit when it doesn&#8217;t know: New tool boosts model transparency (2025, June 3)<br \/>\n                                                retrieved 3 June 2025<br \/>\n                                                from https:\/\/techxplore.com\/<a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>\/2025-06-ai-doesnt-tool-boosts-transparency.html\n                                            <\/p>\n<p>\n                                            This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no<br \/>\n                                            part may be reproduced without the written permission. The content is provided for information purposes only.\n                                            <\/p>\n<\/p><\/div>\n<\/p><\/div>\n<p><script id=\"facebook-jssdk\" async=\"\" src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMN63nwsw68G3Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more Like this articles, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/category\/sciencee\/\" target=\"_blank\" >Science category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/techxplore.com\/news\/2025-06-ai-doesnt-tool-boosts-transparency.html\" target=\"_blank\" >Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Credit: Pixabay\/CC0 Public Domain Artificial intelligence systems like ChatGPT provide plausible-sounding answers to any question you might ask. But they don&#8217;t always reveal the gaps in their knowledge or areas where they&#8217;re uncertain. That problem can have huge consequences as AI systems are increasingly used to do things like develop drugs, synthesize information, and drive&#8230;<\/p>\n","protected":false},"author":1,"featured_media":673341,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/scx2.b-cdn.net\/gfx\/news\/hires\/2024\/ai-and-data-1.jpg","fifu_image_alt":"","footnotes":""},"categories":[16],"tags":[],"class_list":["post-673340","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sciencee"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/673340","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=673340"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/673340\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/673341"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=673340"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=673340"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=673340"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}