{"id":472687,"date":"2022-07-10T23:31:15","date_gmt":"2022-07-10T20:31:15","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/github-copilot-is-the-first-real-product-based-on-large-language-models\/"},"modified":"2022-07-10T23:31:15","modified_gmt":"2022-07-10T20:31:15","slug":"github-copilot-is-the-first-real-product-based-on-large-language-models","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/github-copilot-is-the-first-real-product-based-on-large-language-models\/","title":{"rendered":"#GitHub Copilot is the first real product based on large language models"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a2e3ff57d797\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a2e3ff57d797\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/github-copilot-is-the-first-real-product-based-on-large-language-models\/#%E2%80%9CGitHub_Copilot_is_the_first_real_product_based_on_large_language_models%E2%80%9D\" >&#8220;GitHub Copilot is the first real product based on large language models&#8221;<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/github-copilot-is-the-first-real-product-based-on-large-language-models\/#Greetings_humanoids\" >Greetings, humanoids<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/github-copilot-is-the-first-real-product-based-on-large-language-models\/#Specialized_LLM\" >Specialized LLM<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/github-copilot-is-the-first-real-product-based-on-large-language-models\/#Copilots_productmarket_fit\" >Copilot\u2019s product\/market fit<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/github-copilot-is-the-first-real-product-based-on-large-language-models\/#Distribution_and_cost-efficiency\" >Distribution and cost-efficiency<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"%E2%80%9CGitHub_Copilot_is_the_first_real_product_based_on_large_language_models%E2%80%9D\"><\/span>&#8220;GitHub Copilot is the first real product based on large language models&#8221;<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<div id=\"article-main-content\">\n                            <em>This article is part of our <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/watch-movies-tv-seriess\/\" data-internallinksmanager029f6b8e52c=\"8\" title=\"Watch Movies &amp; TV Series\" target=\"_blank\" rel=\"noopener\">series<\/a> that explores the <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/tag\/ai-business\/\">business of artificial intelligence<\/a><\/em><\/p>\n<p>Since GPT-2, there has been much excitement around the <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>lications of large language models. And in the past few years, we\u2019ve seen LLMs used for many exciting tasks, such as writing articles, designing websites, creating images, and even writing code.<\/p>\n<p>But as I have argued before, there\u2019s a <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2021\/01\/25\/gpt-3-startups-businesses\/\">wide gap<\/a> between showing a new <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" data-internallinksmanager029f6b8e52c=\"4\" title=\"Technology\" target=\"_blank\" rel=\"noopener\">technology<\/a> do something cool and using the same technology to create a successful product with a workable business model.<\/p>\n<div class=\"inarticle-wrapper neural channel-cta hs-embed-tnw\">\n<div id=\"hs-embed-tnw\" class=\"channel-cta-wrapper\">\n<div class=\"channel-cta-img\"><img decoding=\"async\" src=\"https:\/\/s3.amazonaws.com\/uploads.tnw\/uploads\/neural-newsletter_header-1.gif\"\/><\/div>\n<p><noscript><img decoding=\"async\" src=\"https:\/\/thenextweb.com\/news\/src=\" https:=\"\"\/><\/noscript><\/p>\n<div class=\"channel-cta-input\">\n<h2 class=\"channel-cta-title\"><span class=\"ez-toc-section\" id=\"Greetings_humanoids\"><\/span>Greetings, humanoids<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"channel-cta-tagline\">Subscribe to our <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a>letter now for a weekly recap of our favorite AI stories in your inbox.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>Microsoft, I think, just launched the first real LLM product with the <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/github.blog\/2022-06-21-github-copilot-is-generally-available-to-all-developers\/\">public release of GitHub Copilot<\/a> last week. This is an application that has a strong product\/market fit, has immense added value, is hard to beat, is cost-efficient, has very strong distribution channels, and can become a source of great profit.<\/p>\n<p>GitHub Copilot\u2019s release is a reminder of two things: First, LLMs are fascinating, but they are useful when applied to specific tasks as opposed to <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/06\/20\/lamda-large-language-models-sentient-ai\/\">artificial general intelligence<\/a>. And second, the nature of LLMs put large tech companies like Microsoft and Google at <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/05\/16\/opt-175b-large-language-models\/\">an unfair advantage<\/a> to commercialize them\u2014LLMs are not democratic.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Specialized_LLM\"><\/span>Specialized LLM<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" class=\"aligncenter size-featured_img wp-image-1389328 js-lazy\" alt=\"github-copilot-code-generation\" width=\"796\" height=\"448\" sizes=\"auto, (max-width: 796px) 100vw, 796px\" https:=\"\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-796x448.webp 796w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-280x158.webp 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-240x135.webp 240w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-480x270.webp 480w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-1200x675.webp 1200w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation.webp 1392w\"\/><noscript><img loading=\"lazy\" class=\"aligncenter size-featured_img wp-image-1389328\" https:=\"\" alt=\"github-copilot-code-generation\" width=\"796\" height=\"448\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-796x448.webp 796w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-280x158.webp 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-240x135.webp 240w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-480x270.webp 480w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation-1200x675.webp 1200w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-code-generation.webp 1392w\"\/><\/noscript><\/figure>\n<p>Copilot is an <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2021\/07\/05\/openai-github-gpt-3-copilot\/\">AI programming tool<\/a> that is installed as an extension on popular IDEs like Visual Studio and VS Code. It provides suggestions as you write code, something like autocomplete but for programming. Its capabilities range from completing a line of code to creating entire blocks of code such as functions and classes.<\/p>\n<p>Copilot is powered by <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2021\/07\/15\/openai-codex-ai-programming\/\">Codex<\/a>, a version of <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/08\/17\/openai-gpt-3-commercial-ai\/\">OpenAI\u2019s famous GPT-3 model<\/a>, a large language model that made the headlines for its capacity to perform a wide range of tasks. However, contrary to GPT-3, Codex has been finetuned just for programming tasks. And it produces impressive results.<\/p>\n<p>The success of GitHub Copilot and Codex underline one important fact. When it comes to putting LLMs to real use, specialization beats <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a>ization. When Copilot was first introduced in 2021, <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.cnbc.com\/2021\/06\/29\/microsoft-github-copilot-ai-offers-coding-suggestions.html\">CNBC reported<\/a>: \u201c\u2026back when OpenAI was first training [GPT-3], the start-up had no intention of teaching it how to help code, [OpenAI CTO Greg] Brockman said. It was meant more as a <strong>general purpose language model<\/strong> [emphasis mine] that could, for instance, generate articles, fix incorrect grammar and translate from one language into another.\u201d<\/p>\n<p>But while GPT-3 has found mild success in various applications, Copilot and Codex have proven to be great hits in one specific area. Codex can\u2019t write <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/09\/14\/guardian-gpt-3-article-ai-fake-news\/\">poetry or articles like GPT-3<\/a>, but it has proven to be very useful for developers of different levels of expertise. Codex is also much smaller than GPT-3, which means it is more memory and compute efficient. And given that it has been trained for a specific task as opposed to the open-ended and ambiguous world of human language, it is less prone to the <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/06\/27\/large-language-models-logical-reasoning\/\">pitfalls that models like GPT-3 often fall into<\/a>.<\/p>\n<p>It is worth noting, however, that just as <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2021\/12\/20\/artificial-intelligence-large-language-understanding\/\">GPT-3 knows nothing about human language<\/a>, Copilot knows nothing about computer code. It is a <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/05\/02\/what-is-the-transformer\/\">transformer model<\/a> that has been trained on millions of code repositories. Given a prompt (e.g., a piece of code or a textual description), it will try to predict the next sequence of instructions that make the most sense.<\/p>\n<p>With its huge training corpus and massive neural network, Copilot mostly makes good predictions. But sometimes, it might make dumb mistakes that the most novice programmer would avoid. It doesn\u2019t think about programs in the way a programmer does. It can\u2019t design software or think in steps and think about user requirements and experience and all the other things that go into building successful apps. It\u2019s <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2021\/08\/13\/openai-codex-api\/\">not a replacement for human programmers<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Copilots_productmarket_fit\"><\/span>Copilot\u2019s product\/market fit<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" class=\"aligncenter size-featured_img wp-image-1389329 js-lazy\" alt=\"github-copilot-stack-overflow-competition\" width=\"796\" height=\"497\" sizes=\"auto, (max-width: 796px) 100vw, 796px\" https:=\"\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition-796x497.webp 796w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition-280x175.webp 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition-216x135.webp 216w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition-432x270.webp 432w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition.webp 1392w\"\/><noscript><img loading=\"lazy\" class=\"aligncenter size-featured_img wp-image-1389329\" https:=\"\" alt=\"github-copilot-stack-overflow-competition\" width=\"796\" height=\"497\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition-796x497.webp 796w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition-280x175.webp 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition-216x135.webp 216w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition-432x270.webp 432w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/github-copilot-stack-overflow-competition.webp 1392w\"\/><\/noscript><\/figure>\n<p>One of the milestones for any product is achieving product\/market fit, or proving that it can solve some problem better than alternative solutions in the market. In this regard, Copilot has been a stunning success.<\/p>\n<p>GitHub released Copilot as a preview product last June and has since been used by more than one million developers.<\/p>\n<p>According to GitHub, in files where Copilot is activated, it accounts for around an impressive 40 percent of the written code. Developers and engineers <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/venturebeat.com\/2022\/06\/29\/github-copilot-is-now-public-heres-what-you-need-to-know\/\">I spoke to last week<\/a> say that while there are limits to Copilot\u2019s capabilities, there\u2019s no denying that it improves their productivity significantly.<\/p>\n<p>For some use cases, Copilot is competing with StackOverflow and other code forums, where users must search for the solution to a specific problem they face. In this case, the added value of Copilot is very evident and palpable: less frustration and distraction, more focus. Instead of leaving their IDE and searching for a solution on the web, developers just type the description or docstring of the functionality they want, and Copilot does most of the work for them.<\/p>\n<p>In other cases, Copilot is competing against manually writing frustrating code, such as configuring matplotlib charts in Python (a super frustrating task). While Copilot\u2019s might output require some tweaking, it relieves most of the burden on developers.<\/p>\n<p>In many other use cases, Copilot has been able to cement itself as a superior solution to problems that many developers face every day. Developers told me about things such as running test cases, setting up web servers, documenting code, and many other tasks that previously required manual effort and were arduous. Copilot has helped them save a ton of time in their day-to-day work.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Distribution_and_cost-efficiency\"><\/span>Distribution and cost-efficiency<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" class=\"aligncenter size-featured_img wp-image-1389330 js-lazy\" alt=\"GitHub-Copilot-microsoft-openai\" width=\"796\" height=\"448\" sizes=\"auto, (max-width: 796px) 100vw, 796px\" https:=\"\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-796x448.webp 796w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-280x158.webp 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-240x135.webp 240w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-480x270.webp 480w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-1200x675.webp 1200w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai.webp 1392w\"\/><noscript><img loading=\"lazy\" class=\"aligncenter size-featured_img wp-image-1389330\" https:=\"\" alt=\"GitHub-Copilot-microsoft-openai\" width=\"796\" height=\"448\" srcset=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-796x448.webp 796w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-280x158.webp 280w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-240x135.webp 240w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-480x270.webp 480w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai-1200x675.webp 1200w, https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/GitHub-Copilot-microsoft-openai.webp 1392w\"\/><\/noscript><\/figure>\n<p>Product\/market fit is just one of the several components of creating a successful product. If you have a good product but can\u2019t find the right distribution channels to deliver its value in a way that is cost-efficient and profitable, then you\u2019re doomed. At the same time, you\u2019ll need a plan to maintain your edge over competitors, prevent other companies from replicating your success, and make sure that you can continue to deliver value down the stretch.<\/p>\n<p>To turn Copilot into a successful product, Microsoft needed to bring together several very important pieces, including technology, infrastructure, and market.<\/p>\n<p>First, it needed the right technology, which it acquired thanks to its <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/09\/24\/microsoft-openai-gpt-3-license\/\">exclusive license to OpenAI\u2019s technology<\/a>. Since 2019, OpenAI has stopped open-sourcing its technology and is instead licensing it to its financial backers, chief among them Microsoft. Codex and Copilot were created off GPT-3 with the help of OpenAI\u2019s scientists.<\/p>\n<p>Other large tech companies have been able to create large language models that are comparable to GPT-3. But there\u2019s no denying that LLMs are <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/09\/21\/gpt-3-economy-business-model\/\">very costly to train and run<\/a>.<\/p>\n<p>\u201cFor a model that is 10 times smaller than Codex\u2014the model behind Copilot (which has 12B parameters on the paper)\u2014it takes hundreds of dollars to do the evaluation on this <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/huggingface.co\/datasets\/openai_humaneval\">benchmark<\/a> which they used in their paper,\u201d Loubna Ben Allal, machine learning engineer at Hugging Face, told TechTalks. Ben Allal referred to <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/huggingface.co\/datasets\/codeparrot\/apps\">another benchmark<\/a> used for Codex evaluation, which cost thousands of dollars for her own smaller model.<\/p>\n<p>\u201cThere are also security issues because you have to execute untrusted programs to evaluate the model which might be malicious, sandboxes are typically used for security,\u201d Ben Allal said.<\/p>\n<p>Leandro von Werra, another ML engineer at Hugging Face, estimated training costs to be between tens to hundreds of thousands of dollars depending on the size and number of necessary experiments to get it right.<\/p>\n<p>\u201cInference is one of the biggest challenges,\u201d von Werra added in comments to TechTalks. \u201cWhile almost anybody with resources can train a 10B model these days, getting the inference latency low enough to feel responsive to the user is an engineering challenge.\u201d<\/p>\n<p>This is where Microsoft\u2019s second advantage kicks in. The company has been able to create a large cloud infrastructure that is specialized for machine learning models such as Codex. It runs inference and provides suggestions in milliseconds. And more importantly, Microsoft is able to run and provide Copilot at a very affordable price. Currently, Copilot is offered at $10\/month or $100\/year, and it will be provided for free to students and maintainers of popular open-source repositories.<\/p>\n<p>Most developers I spoke to were very satisfied with the pricing model because it made them much more than its price in time saved.<\/p>\n<p>Abhishek Thakur, another ML engineer at Hugging Face I spoke to earlier this week, said, \u201cAs a machine learning engineer, I know that a lot goes into building products like these, especially Copilot, which provides suggestions with sub-milliseconds latency. To build an infrastructure that serves these kinds of models for free is not feasible in the real world for a longer period of time.\u201d<\/p>\n<p>However, running code generator LLMs at affordable rates is not impossible.<\/p>\n<p>\u201cIn terms of the compute to build these models and necessary data: that\u2018s quite feasible and there have been a few replications of Codex such as Incoder from Meta and CodeGen (<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/SFResearch\/status\/1541818596048875520\">now accessible for free on the Hugging Face Hub<\/a>) from Salesforce matching Codex\u2018s performance,\u201d von Werra said. \u201cThere is definitely some engineering involved in building the models into a fast and nice product, but it seems many companies could do this if they want to.\u201d<\/p>\n<p>However, this is where the third piece of the puzzle kicks in. Microsoft\u2019s acquisition of GitHub gave it access to the biggest developer market, making it easy for the company to put Copilot into the hands of millions of users. Microsoft also owns Visual Studio and VS Code, two of the most popular IDEs with hundreds of millions of users. This reduces the friction for developers to adopt Copilot as opposed to another similar product.<\/p>\n<p>With its pricing, efficiency, and market reach, Microsoft seems to have solidified its position as the leader in the emerging market for AI-assisted software development. The market can take other turns. What\u2019s for sure (<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2021\/01\/25\/gpt-3-startups-businesses\/\">and as I have pointed out before<\/a>) is that large language models will open up plenty of opportunities to create new applications and markets. But they won\u2019t change the fundamentals of sound product management.<\/p>\n<div id=\"article-main-content\" class=\"c-richText c-richText--large\">\n<em>This article was originally published by Ben Dickson on<span>\u00a0<\/span><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/\">TechTalks<\/a>, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech, and what we need to look out for. You can read the original article\u00a0<a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2022\/07\/05\/github-copilot-large-language-model-product-management\/\">here<\/a>.<\/em>\n<\/div>\n<\/p><\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/news\/github-copilot-large-language-models\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;GitHub Copilot is the first real product based on large language models&#8221; This article is part of our series that explores the business of artificial intelligence Since GPT-2, there has been much excitement around the applications of large language models. And in the past few years, we\u2019ve seen LLMs used for many exciting tasks, such&#8230;<\/p>\n","protected":false},"author":1,"featured_media":472688,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/copilot.jpg&signature=3d09a4ddf009a22c02f20583ecfd57ce","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-472687","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/472687","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=472687"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/472687\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/472688"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=472687"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=472687"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=472687"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}