{"id":115367,"date":"2020-11-19T12:00:07","date_gmt":"2020-11-19T09:00:07","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/a-beginners-guide-to-ensemble-learning\/"},"modified":"2020-11-19T12:00:07","modified_gmt":"2020-11-19T09:00:07","slug":"a-beginners-guide-to-ensemble-learning","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/a-beginners-guide-to-ensemble-learning\/","title":{"rendered":"#A beginner\u2019s guide to ensemble learning"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a28b79c7284a\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a28b79c7284a\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/a-beginners-guide-to-ensemble-learning\/#How_does_ensemble_learning_work\" >How does ensemble learning work?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/buradabiliyorum.com\/en\/a-beginners-guide-to-ensemble-learning\/#Ensemble_methods\" >Ensemble methods<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/buradabiliyorum.com\/en\/a-beginners-guide-to-ensemble-learning\/#Boosting_methods\" >Boosting methods<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/buradabiliyorum.com\/en\/a-beginners-guide-to-ensemble-learning\/#Random_forests\" >Random forests<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/buradabiliyorum.com\/en\/a-beginners-guide-to-ensemble-learning\/#Challenges_of_ensemble_learning\" >Challenges of ensemble learning<\/a><\/li><\/ul><\/nav><\/div>\n<p>&#8220;<strong>#A beginner\u2019s guide to ensemble learning<\/strong>&#8221;<\/p>\n<div>\n                                The principle of \u201cthe wisdom of the crowd\u201d shows that a large group of people with average knowledge on a topic can provide reliable answers to questions such as predicting quantities, spatial reasoning, and <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a> knowledge. The aggregate results cancel out the noise and can often be superior to those of highly knowledgeable experts. The same rule can <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>ly to artificial intelligence applications that rely on<span>\u00a0<\/span><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2017\/08\/28\/artificial-intelligence-machine-learning-deep-learning\/\">machine learning<\/a>, the branch of AI that predicts outcomes based on mathematical models.<\/p>\n<p>In machine learning, crowd wisdom is achieved through ensemble learning. For many problems, the result obtained from an ensemble, a combination of machine learning models, can be more accurate than any single member of the group.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"How_does_ensemble_learning_work\"><\/span>How does ensemble learning work?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Say you want to develop a machine learning model that predicts inventory stock orders for your company based on historical data you have gathered from previous years. You use train four machine learning models using a different algorithms: linear regression, support vector machine, a regression decision tree, and a basic<span>\u00a0<\/span><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2019\/08\/05\/what-is-artificial-neural-network-ann\/\">artificial neural network<\/a>. But even after much tweaking and configuration, none of them achieves your desired 95 percent prediction accuracy. These machine learning models are called \u201cweak learners\u201d because they fail to converge to the desired level.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\">\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"jetpack-lazy-image jetpack-lazy-image--handled wp-image-8765 lazy\" sizes=\"auto, (max-width: 696px) 100vw, 696px\" alt=\"disparate machine learning models\" width=\"696\" height=\"321\" data-attachment-id=\"8765\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/11\/12\/what-is-ensemble-learning\/disparate-machine-learning-models\/\" data-orig-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?fit=2100%2C968&amp;ssl=1\" data-orig-size=\"2100,968\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"disparate machine learning models\" data-image-description=\"\" data-medium-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?fit=300%2C138&amp;ssl=1\" data-large-file=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?fit=696%2C321&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\" src=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=696%2C321&amp;ssl=1\" data-lazy=\"true\" srcset=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=1024%2C472&amp;ssl=1 1024w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=300%2C138&amp;ssl=1 300w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=768%2C354&amp;ssl=1 768w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=1536%2C708&amp;ssl=1 1536w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=2048%2C944&amp;ssl=1 2048w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=696%2C321&amp;ssl=1 696w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=1068%2C492&amp;ssl=1 1068w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=911%2C420&amp;ssl=1 911w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?resize=1920%2C885&amp;ssl=1 1920w, https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/disparate-machine-learning-models.jpg?w=1392&amp;ssl=1 1392w\"\/><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/11\/19\/a-beginners-guide-to-ensemble-learning-syndication\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F11%2F19%2Fa-beginners-guide-to-ensemble-learning-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: Single machine learning models do not provide the desired accuracy\" data-title=\"Share Single machine learning models do not provide the desired accuracy on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share Single machine learning models do not provide the desired accuracy on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>Single machine learning models do not provide the desired accuracy<\/figcaption><\/figure>\n<\/p>\n<\/figure>\n<\/div>\n<p>But weak doesn\u2019t mean useless. You can combine them into an ensemble. For each new prediction, you run your input data through all four models, and then compute the average of the results. When examining the new result, you see that the aggregate results provide 96 percent accuracy, which is more than acceptable.<\/p>\n<p>The reason ensemble learning is efficient is that your machine learning models work differently. Each model might perform well on some data and less accurately on others. When you combine all them, they cancel out each other\u2019s weaknesses.<\/p>\n<p>You can apply ensemble methods to both predictions problems, like the inventory prediction example we just saw, and classification problems, such as determining whether a picture contains a certain object.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\">\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"jetpack-lazy-image jetpack-lazy-image--handled wp-image-8766 lazy\" sizes=\"auto, (max-width: 696px) 100vw, 696px\" alt=\"ensemble machine learning models\" width=\"696\" height=\"392\" data-attachment-id=\"8766\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/11\/12\/what-is-ensemble-learning\/ensemble-machine-learning-models\/\" data-orig-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?fit=2560%2C1440&amp;ssl=1\" data-orig-size=\"2560,1440\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"ensemble machine learning models\" data-image-description=\"\" data-medium-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?fit=300%2C169&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?fit=696%2C392&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=696%2C392&amp;ssl=1\" data-lazy=\"true\" srcset=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=1024%2C576&amp;ssl=1 1024w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=300%2C169&amp;ssl=1 300w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=768%2C432&amp;ssl=1 768w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=1536%2C864&amp;ssl=1 1536w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=2048%2C1152&amp;ssl=1 2048w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=696%2C392&amp;ssl=1 696w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=1068%2C601&amp;ssl=1 1068w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=747%2C420&amp;ssl=1 747w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?resize=1920%2C1080&amp;ssl=1 1920w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/ensemble-machine-learning-models.jpg?w=1392&amp;ssl=1 1392w\"\/><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/11\/19\/a-beginners-guide-to-ensemble-learning-syndication\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F11%2F19%2Fa-beginners-guide-to-ensemble-learning-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: Ensemble machine learning combine several models to improve the overall results.\" data-title=\"Share Ensemble machine learning combine several models to improve the overall results. on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share Ensemble machine learning combine several models to improve the overall results. on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>Ensemble machine learning combine several models to improve the overall results.<\/figcaption><\/figure>\n<\/p>\n<\/figure>\n<\/div>\n<h2><span class=\"ez-toc-section\" id=\"Ensemble_methods\"><\/span>Ensemble methods<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>For a machine learning ensemble, you must make sure your models are independent of each other (or as independent of each other as possible). One way to do this is to create your ensemble from different algorithms, as in the above example.<\/p>\n<p>Another ensemble method is to use instances of the same machine learning algorithms and train them on different data sets. For instance, you can create an ensemble composed of 12 linear regression models, each trained on a subset of your training data.<\/p>\n<p>There are two key methods for sampling data from your training set. \u201cBootstrap aggregation,\u201d aka \u201cbagging,\u201d takes random samples from the training set \u201cwith replacement.\u201d The other method, \u201cpasting,\u201d draws samples \u201cwithout replacement.\u201d<\/p>\n<p>To understand the difference between the sampling methods, here\u2019s an example. Say you have a training set with 10,000 samples and you want to train each machine learning model in your ensemble with 9,000 samples. In case you\u2019re using bagging, for each of your machine learning models, you take the following steps:<\/p>\n<ol type=\"1\">\n<li>Draw a random sample from the training set.<\/li>\n<li>Add a copy of the sample to the model\u2019s training set<\/li>\n<li>Return the sample to the original training set<\/li>\n<li>Repeat the process 8,999 times<\/li>\n<\/ol>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\">\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"jetpack-lazy-image jetpack-lazy-image--handled wp-image-8767 lazy\" sizes=\"auto, (max-width: 666px) 100vw, 666px\" alt=\"bagging sampling\" width=\"666\" height=\"539\" data-attachment-id=\"8767\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/11\/12\/what-is-ensemble-learning\/bagging-sampling\/\" data-orig-file=\"https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?fit=1780%2C1440&amp;ssl=1\" data-orig-size=\"1780,1440\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"bagging sampling\" data-image-description=\"\" data-medium-file=\"https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?fit=300%2C243&amp;ssl=1\" data-large-file=\"https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?fit=696%2C563&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\" src=\"https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?resize=666%2C539&amp;ssl=1\" data-lazy=\"true\" srcset=\"https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?resize=1024%2C828&amp;ssl=1 1024w, https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?resize=300%2C243&amp;ssl=1 300w, https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?resize=768%2C621&amp;ssl=1 768w, https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?resize=1536%2C1243&amp;ssl=1 1536w, https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?resize=696%2C563&amp;ssl=1 696w, https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?resize=1068%2C864&amp;ssl=1 1068w, https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?resize=519%2C420&amp;ssl=1 519w, https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?w=1780&amp;ssl=1 1780w, https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/bagging-sampling.jpg?w=1392&amp;ssl=1 1392w\"\/><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/11\/19\/a-beginners-guide-to-ensemble-learning-syndication\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F11%2F19%2Fa-beginners-guide-to-ensemble-learning-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: Bagging sampling draws samples from the training set and replaces them\" data-title=\"Share Bagging sampling draws samples from the training set and replaces them on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share Bagging sampling draws samples from the training set and replaces them on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>Bagging sampling draws samples from the training set and replaces them<\/figcaption><\/figure>\n<\/p>\n<\/figure>\n<\/div>\n<p>When using pasting, you go through the same process, with the difference that samples are not returned to the training set after being drawn. Consequently, the same sample might appear in a model\u2019s several times when using bagging but only once when using pasting.<\/p>\n<p>After training all your machine learning models, you\u2019ll have to choose an aggregation method. If you\u2019re tackling a classification problem, the usual aggregation method is \u201cstatistical mode,\u201d or the class that is predicted more than others. In regression problems, ensembles usually use the average of the predictions made by the models.<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\">\n<figure class=\"post-image post-mediaBleed aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"jetpack-lazy-image jetpack-lazy-image--handled wp-image-8768 lazy\" sizes=\"auto, (max-width: 696px) 100vw, 696px\" alt=\"pasting sampling\" width=\"696\" height=\"562\" data-attachment-id=\"8768\" data-permalink=\"https:\/\/bdtechtalks.com\/2020\/11\/12\/what-is-ensemble-learning\/pasting-sampling\/\" data-orig-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?fit=1780%2C1440&amp;ssl=1\" data-orig-size=\"1780,1440\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"pasting sampling\" data-image-description=\"\" data-medium-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?fit=300%2C243&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?fit=696%2C563&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?resize=696%2C562&amp;ssl=1\" data-lazy=\"true\" srcset=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?resize=1024%2C828&amp;ssl=1 1024w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?resize=300%2C243&amp;ssl=1 300w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?resize=768%2C621&amp;ssl=1 768w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?resize=1536%2C1243&amp;ssl=1 1536w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?resize=696%2C563&amp;ssl=1 696w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?resize=1068%2C864&amp;ssl=1 1068w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?resize=519%2C420&amp;ssl=1 519w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?w=1780&amp;ssl=1 1780w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2020\/11\/pasting-sampling.jpg?w=1392&amp;ssl=1 1392w\"\/><figcaption><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/thenextweb.com\/neural\/2020\/11\/19\/a-beginners-guide-to-ensemble-learning-syndication\/#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fthenextweb.com%2Fneural%2F2020%2F11%2F19%2Fa-beginners-guide-to-ensemble-learning-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: Pasting draws samples from the training set and replaces them\" data-title=\"Share Pasting draws samples from the training set and replaces them on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share Pasting draws samples from the training set and replaces them on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"\/><\/a>Pasting draws samples from the training set and replaces them<\/figcaption><\/figure>\n<\/p>\n<\/figure>\n<\/div>\n<h2><span class=\"ez-toc-section\" id=\"Boosting_methods\"><\/span>Boosting methods<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Another popular ensemble technique is \u201cboosting.\u201d In contrast to classic ensemble methods, where machine learning models are trained in parallel, boosting methods train them sequentially, with each new model building up on the previous one and solving its inefficiencies.<\/p>\n<p>AdaBoost (short for \u201cadaptive boosting\u201d), one of the more popular boosting methods, improves the accuracy of ensemble models by adapting new models to the mistakes of previous ones. After training your first machine learning model, you single out the training examples misclassified or wrongly predicted by the model. When training the next model, you put more emphasis on these examples. This results in a machine learning model that performs better where the previous one failed. The process repeats itself for as many models you want to add to the ensemble. The final ensemble contains several machine learning models of different accuracies, which together can provide better accuracy. In boosted ensembles, the output of each model is given a weight that is proportionate to its accuracy.<br \/>\n<iframe loading=\"lazy\" title=\"Extending Machine Learning Algorithms \u2013 AdaBoost Classifier | packtpub.com\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/BoGNyWW9-mE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Random_forests\"><\/span>Random forests<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>One area where ensemble learning is very popular is decision trees, a machine learning algorithm that is very useful because of its flexibility and interpretability. Decision trees can make predictions on complex problems, and they can also trace back their outputs to a <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/watch-movies-tv-seriess\/\" data-internallinksmanager029f6b8e52c=\"8\" title=\"Watch Movies &amp; TV Series\" target=\"_blank\" rel=\"noopener\">series<\/a> of very clear steps.<\/p>\n<p>The problem with decision trees is that they don\u2019t create smooth boundaries between different classes unless you break them down into too many branches, in which case they become prone to \u201coverfitting,\u201d a problem that occurs when a machine learning model performs very well on training data but poorly on novel examples from the real world.<\/p>\n<p>This is a problem that can be solved through ensemble learning. Random forests are machine learning ensembles composed of multiple decision trees (hence the name \u201cforest\u201d). Using random forests ensures that a machine learning model does not get caught up in the specific confines of a single decision tree.<\/p>\n<p>Random forests have their own independent implementation in Python machine learning libraries such as scikit-learn.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Challenges_of_ensemble_learning\"><\/span>Challenges of ensemble learning<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-6223 jetpack-lazy-image jetpack-lazy-image--handled\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=696%2C392&amp;ssl=1\" sizes=\"auto, (max-width: 696px) 100vw, 696px\" srcset=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=1024%2C576&amp;ssl=1 1024w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=300%2C169&amp;ssl=1 300w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=768%2C432&amp;ssl=1 768w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=1536%2C864&amp;ssl=1 1536w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=2048%2C1152&amp;ssl=1 2048w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=696%2C392&amp;ssl=1 696w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=1068%2C601&amp;ssl=1 1068w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=747%2C420&amp;ssl=1 747w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?resize=1920%2C1080&amp;ssl=1 1920w, https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?w=1392&amp;ssl=1 1392w\" alt=\"random vectors\" width=\"696\" height=\"392\" data-attachment-id=\"6223\" data-permalink=\"https:\/\/bdtechtalks.com\/random-vectors\/\" data-orig-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?fit=3840%2C2160&amp;ssl=1\" data-orig-size=\"3840,2160\" data-comments-opened=\"1\" data-image-meta=\"{\" aperture=\"\" data-image-title=\"random vectors\" data-image-description=\"\" data-medium-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?fit=300%2C169&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/12\/random-vectors.jpg?fit=696%2C392&amp;ssl=1\" data-recalc-dims=\"1\" data-lazy-loaded=\"1\"\/><\/figure>\n<p>While ensemble learning is a very powerful tool, it also has some tradeoffs.<\/p>\n<p>Using ensemble means you must spend more time and resources on training your machine learning models. For instance, a random forest with 500 trees provides much better results than a single decision tree, but it also takes much more time to train. Running ensemble models can also become problematic if the algorithms you use require a lot of memory.<\/p>\n<p>Another problem with ensemble learning is<span>\u00a0<\/span><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2018\/09\/25\/explainable-interpretable-ai\/\">explainability<\/a>. While adding new models to an ensemble can improve its overall accuracy, it makes it harder to investigate the decisions made by the AI algorithm. A single machine learning models such as decision tree is easy to trace, but when you have hundreds of models contributing to an output, it is much more difficult to make sense of the logic behind each decision.<\/p>\n<p>As with most everything you\u2019ll encounter in machine learning, ensemble is one among the many tools you have for solving complicated problems. It can get you out of difficult situations, but it\u2019s not a silver bullet. Use it wisely.<\/p>\n<hr\/>\n<p><i><span style=\"font-weight: 400;\">This article was originally published by Ben Dickson on <\/span><\/i><a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/\"><i><span style=\"font-weight: 400;\">TechTalks<\/span><\/i><\/a><i><span style=\"font-weight: 400;\">, a publication that examines trends in <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/technology\/\" data-internallinksmanager029f6b8e52c=\"4\" title=\"Technology\" target=\"_blank\" rel=\"noopener\">technology<\/a>, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech and what we need to look out for. You can read the original article <a rel=\"nofollow noopener noreferrer\" target=\"_blank\" href=\"https:\/\/bdtechtalks.com\/2020\/11\/12\/what-is-ensemble-learning\/\">here<\/a>.<\/span><\/i><\/p>\n<p class=\"c-post-pubDate\">\n                                    Published November 19, 2020 \u2014 09:00 UTC<\/p><\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><script data-src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js#xfbml=1&amp;appId=378011798897423&amp;version=v2.6\" id=\"socialSrcFacebook\" type=\"text\/template\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more like this article, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/technology\/\" target=\"_blank\" rel=\"noopener noreferrer\">Technology category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/thenextweb.com\/neural\/2020\/11\/19\/a-beginners-guide-to-ensemble-learning-syndication\/\" target=\"_blank\" rel=\"noopener noreferrer\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#A beginner\u2019s guide to ensemble learning&#8221; The principle of \u201cthe wisdom of the crowd\u201d shows that a large group of people with average knowledge on a topic can provide reliable answers to questions such as predicting quantities, spatial reasoning, and general knowledge. The aggregate results cancel out the noise and can often be superior to&#8230;<\/p>\n","protected":false},"author":1,"featured_media":115368,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/img-cdn.tnwcdn.com\/image\/neural?filter_last=1&fit=1280,640&url=https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2020\/11\/Untitled-design.png&signature=4ed41725e8d8cb8e8aad5ee76cb18bda","fifu_image_alt":"","footnotes":""},"categories":[18],"tags":[],"class_list":["post-115367","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/115367","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=115367"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/115367\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/115368"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=115367"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=115367"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=115367"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}