{"id":72138,"date":"2020-09-21T20:52:39","date_gmt":"2020-09-21T17:52:39","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/ai-devs-created-a-lean-mean-gpt-3-beating-machine-that-uses-99-9-fewer-parameters\/"},"modified":"2020-09-21T20:52:39","modified_gmt":"2020-09-21T17:52:39","slug":"ai-devs-created-a-lean-mean-gpt-3-beating-machine-that-uses-99-9-fewer-parameters","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/ai-devs-created-a-lean-mean-gpt-3-beating-machine-that-uses-99-9-fewer-parameters\/","title":{"rendered":"#AI devs created a lean, mean, GPT-3-beating machine that uses 99.9% fewer parameters"},"content":{"rendered":"

“#AI devs created a lean, mean, GPT-3-beating machine that uses 99.9% fewer parameters<\/strong>”<\/p>\n

\n

If you want to watch Movies<\/a> or TV series visit the Dizi.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n

<\/p>\n

\n AI researchers from the Ludwig Maximilian University (LMU) of Munich have developed a bite-sized text generator capable of besting OpenAI\u2018s state of the art GPT-3 using only a tiny fraction of its parameters.<\/p>\n

GPT-3 is a monster of an AI system capable of responding to almost any text prompt with unique, original responses that are often surprisingly cogent. It\u2019s an example of what incredibly talented developers can do with cutting-edge algorithms and software when given unfettered access to supercomputers.<\/p>\n

But it\u2019s not very efficient. At least not when compared to a new system developed by LMU researchers Timo Schick and Hinrich Schutze.<\/p>\n

[Read: OpenAI reveals the pricing plans for its API \u2014 and it ain\u2019t cheap]<\/em><\/p>\n

According to a recent pre-print paper<\/a> on arXiv, the duo\u2019s system outperforms GPT-3 on the \u201csuperGLUE\u201d benchmark test with only 223 million parameters:<\/p>\n

In this work, we show that performance similar to GPT-3 can be obtained with language models whose parameter count is several orders of magnitude smaller. This is achieved by converting textual inputs into cloze questions that contain some form of task descript<\/a>ion, combined with gradient-based optimization; additionally exploiting unlabeled data gives further improvements.<\/p>\n<\/blockquote>\n

Parameters are variables used to tune and tweak AI models. They\u2019re intimated from data \u2013 in essence the more parameters an AI model is trained with, the more robust we expect it to be.<\/p>\n

When a system using 99.9% less model parameters is able to best the best at a benchmark task, it\u2019s a pretty big deal. This isn\u2019t to say that the LMU system is better than GPT-3, nor that it\u2019s capable of beating it in tests other than the SuperGLUE benchmark \u2013 which isn\u2019t indicative of GPT-3\u2019s overall capabilities.<\/p>\n

The LMU system\u2019s results come courtesy of a training method called pattern-exploiting training (PET). According to Open AI policy director Jack Clark, writing in the weekly ImportAI newsletter<\/a>:<\/p>\n

Their approach fuses a training technique called PET (pattern-exploiting training) with a small pre-trained Albert model, letting them create a system that \u201coutperform GPT-3 on SuperGLUE with 32 training examples, while requiring only 0.1% of its parameters.\u201d<\/p>\n<\/blockquote>\n

Clark goes on to point out that, while it won\u2019t outperform GPT-3 in every task, it does open new avenues for researchers looking to push the boundaries of AI with more modest hardware.<\/p>\n

For more information check out the duo\u2019s paper here<\/a>.<\/p>\n

H\/t:\u00a0Jack Clark<\/a> and ImportAI<\/a><\/em><\/p>\n

So you\u2019re interested in AI? Then\u00a0<\/span>join our online event, TNW2020<\/span>, where you\u2019ll hear how artificial intelligence is transforming industries and businesses.<\/span><\/em><\/p>\n

\n Published September 21, 2020 \u2014 17:52 UTC<\/p><\/div>\n