{"id":652373,"date":"2025-02-05T23:55:15","date_gmt":"2025-02-05T20:55:15","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/why-iq-is-a-poor-test-for-ai\/"},"modified":"2025-02-05T23:55:15","modified_gmt":"2025-02-05T20:55:15","slug":"why-iq-is-a-poor-test-for-ai","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/why-iq-is-a-poor-test-for-ai\/","title":{"rendered":"#Why IQ is a poor test for AI"},"content":{"rendered":"
\n

During a recent press appearance<\/a>, OpenAI CEO Sam Altman said that he\u2019s observed the \u201cIQ\u201d of AI rapidly improve over the past several years.<\/p>\n

\u201cVery roughly, it feels to me like \u2014 this is not scientifically accurate, this is just a vibe or spiritual answer \u2014 every year we move one standard deviation of IQ,\u201d Altman said. <\/p>\n

Altman isn\u2019t the first to use IQ, an estimation of a person\u2019s intelligence, as a benchmark for AI progress. AI influencers<\/a> on social media<\/a> have given models IQ tests and ranked the results. <\/p>\n

But many experts say that IQ is a poor measure of a model\u2019s capabilities \u2014 and a misleading one.<\/p>\n

\u201cIt can be very tempting to use the same measures we use for humans to describe capabilities or progress, but this is like comparing app<\/a>les with oranges,\u201d Sandra Wachter, a researcher studying tech and regulation at Oxford, told TechCrunch. <\/p>\n

In his comments at the presser, Altman equated IQ with intelligence. Yet IQ tests are relative \u2014 not objective \u2014 measures of certain <\/em>kinds of intelligence. There\u2019s some<\/a> consensus<\/a> that IQ is a reasonable test of logic and abstract reasoning. But it doesn\u2019t measure practical<\/em> intelligence \u2014 knowing how to make things work \u2014 and it\u2019s at best a snapshot.<\/p>\n

\u201cIQ\u00a0is a tool to measure human capabilities \u2014 a contested one no less \u2014 based on what scientists believe human intelligence looks like,\u201d Wachter noted. \u201cBut you can\u2019t use the same measure to describe AI capabilities. A car is faster than humans, and a submarine is better at diving. But this doesn\u2019t mean cars or submarines surpass human intelligence. You\u2019re equivocating one aspect of performance with human intelligence, which is much more complex.\u201d<\/p>\n

To excel at an IQ test, the origins of which some historians<\/a> trace back to eugenics, the widely discredited scientific theory that people can be improved through selective breeding, a test taker must have a strong working memory and knowledge of Western cultural norms<\/a>. This invites the opportunity for bias, of course, which is why one psychologist has called IQ tests<\/a> \u201cideologically corruptible mechanical models\u201d of intelligence. <\/p>\n

That a model might do well on an IQ test indicates more about the test\u2019s flaws than the model\u2019s performance, according to Os Keyes, a doctorate candidate at the University of Washington studying ethical AI.<\/p>\n

\u201c[These] tests are pretty easy to game<\/a> if you have a practically infinite amount of memory and patience,\u201d Keyes said. \u201cIQ tests are a highly limited way of measuring cognition, sentience, and intelligence, something we\u2019ve known since before the invention of the digital computer itself.\u201d<\/p>\n

AI likely has an unfair advantage on IQ tests, as well, considering that models have massive amounts of memory and internalized knowledge at their disposal. Often, models are trained on public web data, and the web is full of example questions taken from IQ tests. <\/p>\n

\u201cTests tend to repeat very similar patterns \u2014 a pretty foolproof way to raise your IQ is to practice taking IQ tests, which is essentially what every [model] has done,\u201d said Mike Cook, a research fellow at King\u2019s College London specializing in AI. \u201cWhen I learn something, I don\u2019t get it piped into my brain with perfect clarity 1 million times, unlike AI, and I can\u2019t process it with no noise or signal loss, either.\u201d<\/p>\n

Ultimately, IQ tests \u2014 biased as they are \u2014 were designed for humans, Cook added \u2014 intended as a way to evaluate general<\/a> problem-solving abilities. They\u2019re inappropriate for a technology<\/a> that approaches solving problems in a very different way than people do.<\/p>\n

\u201cA crow might be able to use a tool to recover a treat from a box, but that doesn\u2019t mean it can enroll at Harvard,\u201d Cook said. \u201cWhen I solve a mathematics problem, my brain is also contending with its ability to read the words on the page correctly, to not think about the shopping I need to do on the way home, or if it\u2019s too cold in the room right now. In other words, human brains contend with a lot more things when they solve a problem \u2014 any problem at all, IQ tests or otherwise \u2014 and they do it with a lot less help [than AI.]\u201d<\/p>\n

All this points to the need for better AI tests, Heidy Khlaaf, chief AI scientist at the AI Now Institute, told TechCrunch. <\/p>\n

\u201cIn the history of computation, we haven\u2019t compared computing abilities to that of humans\u2019 precisely because the nature of computation means systems have always been able to complete tasks already beyond human ability,\u201d Khlaaf said. \u201cThis idea that we directly compare systems\u2019 performance against human abilities is a recent phenomenon that is highly contested, and what surrounds the controversy\u00a0of the ever-expanding \u2014 and moving \u2014 benchmarks being created to evaluate AI systems.\u201d<\/p>\n<\/div>\n

If you liked the article, do not forget to share it with your friends. Follow us on\u00a0Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n

\n

If you want to read more like this article, you can visit our Technology<\/a><\/span> category.<\/strong><\/p>\n<\/blockquote>\n

Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

During a recent press appearance, OpenAI CEO Sam Altman said that he\u2019s observed the \u201cIQ\u201d of AI rapidly improve over the past several years. \u201cVery roughly, it feels to me like \u2014 this is not scientifically accurate, this is just a vibe or spiritual answer \u2014 every year we move one standard deviation of IQ,\u201d…<\/p>\n","protected":false},"author":1,"featured_media":652374,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/techcrunch.com\/wp-content\/uploads\/2023\/06\/GettyImages-1474076387.jpg?resize=1200,768","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[77337,153717,154170,96327],"class_list":["post-652373","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology","tag-ai","tag-ai-benchmarks","tag-iq-tests","tag-performance"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/652373","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=652373"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/652373\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/652374"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=652373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=652373"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=652373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}