{"id":317370,"date":"2021-08-06T21:46:47","date_gmt":"2021-08-06T18:46:47","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/federal-election-2021-know-your-pollsters\/"},"modified":"2021-08-06T21:46:47","modified_gmt":"2021-08-06T18:46:47","slug":"federal-election-2021-know-your-pollsters","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/federal-election-2021-know-your-pollsters\/","title":{"rendered":"#Federal election 2021: Know your pollsters"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_84 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a2554e4aaebc\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #dd3333;color:#dd3333\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #dd3333;color:#dd3333\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a2554e4aaebc\" checked aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/buradabiliyorum.com\/en\/federal-election-2021-know-your-pollsters\/#In_conclusion\" >In conclusion<\/a><\/li><\/ul><\/nav><\/div>\n<p>&#8220;<strong>#Federal election 2021: Know your pollsters<\/strong>&#8221;<\/p>\n<div>\n                                                                        In his best-selling book <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/www.chapters.indigo.ca\/en-ca\/books\/cracking-the-quebec-code-the\/9781988002361-item.html\"><i>Cracking the Quebec Code<\/i><\/a>, Jean-Marc L\u00e9ger wrote\u00a0\u201cPolitical polls make for about 1% of all my revenues, but account to 99% of my problems.\u201d<\/p>\n<p>So, why do it then? For many market-research firms, political polling serves as an efficient way to promote their brand, attract customers and, when successful, show clients their numbers can be trusted.\u00a0In a functioning supply-and-demand market, polling firms whose results are constantly and systematically off the mark are doomed to fail in the long run, not dissimilar to Darwinian economics. (Nevertheless, it should be stressed that even the best polling firms <i>occasionally<\/i> miss the mark\u2014such is the nature of statistics, and the challenge of aiming at a constantly moving target.)<\/p>\n<p>\u201cBut there is more to polling than voting intentions,\u201d I have been told many times, and rightly so. Polling on <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/social-mediaa\/\" data-internallinksmanager029f6b8e52c=\"1\" title=\"Social Media\" target=\"_blank\" rel=\"noopener\">social<\/a> issues can be far more important to understand the political landscape than merely measuring the horserace.\u00a0 However, the reason I do not pay much attention to polls that do not include voting intentions is, simply, that their results are not <i>verifiable<\/i>.<\/p>\n<p>For instance, imagine a market research firm X that publishes a poll indicating, say, that 80 per cent of Canadians are in favour of stricter gun control, or that 25 per cent of Canadians will not take a COVID-19 vaccine under any circumstance. How does one know whether the data is correctly calibrated? How do we know the results aren\u2019t skewed, even involuntarily so?<\/p>\n<p>Now, say, hypothetically, that same poll measures the Conservatives leading the Liberals by 25 points nationally. Given that the current <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/338canada.com\/\">338Canada projections<\/a>\u00a0(based on a weighted rolling average of polls) show the Liberals ahead by a margin of 4 to 8 points, how much traction and\/or media attention should this poll be given, especially the results from its questions on social issues? (\u201cVery little to none\u201d would be the correct answer.) Voting intentions are the only data points that are actually verifiable by experience.<\/p>\n<p>Additionally, since close to a dozen firms regularly poll federal voting intentions in this country, we can compare the results between those firms and keep track of all the numbers over time.<\/p>\n<p>Because elections do occur from time to time, and they represent the perfect opportunity to compare poll results with election results. We may then compile every firm\u2019s report card (see 338Canada\u2019s <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/338canada.com\/pollster-ratings.htm\">ratings of Canadian pollsters here<\/a>). Which firm was right, which one was off? By how much? And which party did the results lean towards? Is the error <i>systematic<\/i> or does it fluctuate from one party to the other?<\/p>\n<p>We present today graphs and data to measure, if any, the \u201cHouse Effect\u201d of Canadian polling firms in federal polls. This analysis will focus on the voting intentions for the federal Liberals and Conservatives spanning the last three years, from January 2018 to July 2021 (and we\u2019ll analyze numbers from other parties at a later date).<\/p>\n<p>Here is a \u201cbullseye graph\u201d that compares <i>all<\/i> the federal polls during that time period with the rolling 338Canada average (Liberals horizontally, Conservatives vertically).<\/p>\n<p><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/338Canada.com\/polls.htm\"><br \/><img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-08-01-be.png\" width=\"680\" vspace=\"18\"\/><br \/><\/a><\/p>\n<p>The scales of the graph represent the number of percentage points off the average at the time of each poll\u2019s middle field date.<\/p>\n<p>As you may expect, the bullseye is surrounded by dots spread in a near-symmetrical fashion. In the long run, the better calibrated and accurate firms should have polls spread almost evenly around the bullseye. In short: Polls near the bullseye were close to the weighted averages during their field dates; those far away from the centre are either statistical outliers or precursors of new trends. (All the polls used in this analysis are listed on <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/338canada.com\/polls.htm\">this page<\/a>.)<\/p>\n<p>Let us look at the same graph, but with added coloured dots to identify polls <i>by firm<\/i>. Here is the graph for L\u00e9ger (green dots):<br \/>\n<img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-07-25-leger.png\" width=\"680\" vspace=\"18\"\/>As you can see, L\u00e9ger polls are spread almost symmetrically along the upper-left to lower-right diagonal. A few L\u00e9ger polls were several points off the average at the time of their field dates, but most dots are located near the bullseye. This is a well calibrated graph.<\/p>\n<p>Here is the same graph for Nanos Research:<br \/>\n<img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-07-25-nanos.png\" width=\"680\" vspace=\"18\"\/>Of note: most of Nanos Research polls are weekly rolling polls, so only one poll per four-week cycles are depicted on the graph. Once again, we notice the dots are nearly evenly spread.<\/p>\n<p>Here is the graph for Ipsos:<br \/>\n<img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-07-25-ipsos.png\" width=\"680\" vspace=\"18\"\/>As with L\u00e9ger and Nanos Research, Ipsos polls are neatly spread around the bullseye with no <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/download-scripts-themes-apps\/\" data-internallinksmanager029f6b8e52c=\"9\" title=\"Download Scripts &amp; Themes &amp; Apps\" target=\"_blank\" rel=\"noopener\">app<\/a>arent, systematic skew.<\/p>\n<p>Here is the graph for Abacus Data:<br \/>\n<img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-07-25-aba.png\" width=\"680\" vspace=\"18\"\/>Same story here. Although there are slightly more polls on the lower half of the graph (hence, lower CPC numbers), we see no major skew in the data.<\/p>\n<p>Here is the graph for Mainstreet Research:<br \/>\n<img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-07-25-ms.png\" width=\"680\" vspace=\"18\"\/>Mainstreet Research\u2019s federal polls appear to have a slight tilt toward the upper-right quadrant of the graph, meaning high LPC and high CPC numbers than the rolling average (hence lower NDP). However, averaging those out (as we will show below), we measure no major skew.<\/p>\n<p>Here is the graph for Innovative Research Group (IRG):<br \/>\n<img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-07-25-ir.png\" width=\"680\" vspace=\"18\"\/>Regular readers of this column and polling nerds may have noticed that IRG polls often seem to be favourable to the Liberals, and that impression would be backed by the data. As you may see on the graph, polls from IRG are almost all located in the lower-right quadrant, meaning high LPC and low CPC numbers. This is a clear example of a systematic house effect. From a purely statistical point of view, it simply cannot be a coincidence that all these dots lean towards the same direction. It <b>does not<\/b> mean that IRG polls are wrong or unusable, but we must use caution and interpret them in the context.<\/p>\n<p>Finally, here is the graph for the Angus Reid Institute (ARI):<br \/>\n<img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-07-25-ari.png\" width=\"680\" vspace=\"18\"\/>We see here the opposite tendency of IRG: Virtually all of Angus Reid\u2019s polls are located in the upper-left quadrant, meaning high CPC and low LPC numbers compared to the poll average. This also shows a systematic house effect. Interestingly though, this measurable tilt towards the federal conservatives does not translate into a noticeable skew in Angus Reid\u2019s provincial polls.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"In_conclusion\"><\/span>In conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>By calculating the average skew from each firm that regularly polls federal voting intentions, we get the following graph:<\/p>\n<p><a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/338Canada.com\/polls.htm\"><br \/><img decoding=\"async\" data-sizes=\"auto\" src=\"https:\/\/qc125.com\/proj\/2021-08-01-be3.png\" width=\"680\" vspace=\"18\"\/><br \/><\/a><\/p>\n<p>Eight of 10 firms depicted on the graph are located within the 2-point radius, which, all things considered, should be seen as a minor skew over the past three years. As for Innovative and Angus Reid, they appear to mirror each other from opposing quadrants.<\/p>\n<p>\u201cBut what if either those firms are correct and everybody else is wrong?\u201d In the 2019 federal election, the final poll from Angus Reid had the Conservatives winning by 5 points over the Liberals. As for Innovative Research, its final numbers showed a two-point win vote for the Liberals. The Conservatives ended up winning the popular vote by one point. Angus Reid had the LPC a little too low, and IRG had the CPC a little too low. The overall numbers were fairly accurate, but they skewed in the same <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/general\/\" data-internallinksmanager029f6b8e52c=\"3\" title=\"General\" target=\"_blank\" rel=\"noopener\">general<\/a> direction as depicted above.<\/p>\n<p>Again, I want to emphasize this point: It is not that their polls are necessarily wrong or unusable. They generally observe the same tendencies as their competitors, which is why their work still has tremendous value (and are regularly and rightly quoted by <a href=\"https:\/\/buradabiliyorum.com\/en\/category\/news\/\" data-internallinksmanager029f6b8e52c=\"2\" title=\"News\" target=\"_blank\" rel=\"noopener\">news<\/a> outlets across the country, including this magazine). But while they see the tide moving the same way as other polling firms, they appear to measure it from a slightly different water level.<\/p>\n<p>Follow <a rel=\"nofollow noopener\" target=\"_blank\" href=\"https:\/\/twitter.com\/338Canada\">338Canada on Twitter<\/a><br \/>\n<span class=\"ctx-article-root\"><!-- --><\/span><\/p><\/div>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><script async defer crossorigin=\"anonymous\" src=\"https:\/\/connect.facebook.net\/en_US\/sdk.js#xfbml=1&#038;version=v10.0\"><\/script><\/p>\n<blockquote><p><strong><span style=\"color: #ff6600;\">If you liked the article, do not forget to share it with your friends. Follow us on\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" href=\"https:\/\/news.google.com\/publications\/CAAqBwgKMLG0nwswvr63Aw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Google News<\/a><\/span>\u00a0too, click on the star and choose us from your favorites.<\/span><\/strong><\/p><\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\">For forums sites go to <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/forum.buradabiliyorum.com\/\" target=\"_blank\" rel=\"noopener\">Forum.BuradaBiliyorum.Com<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<blockquote>\n<p style=\"text-align: center;\"><strong>If you want to read more News articles, you can visit our <span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" href=\"https:\/\/en.buradabiliyorum.com\/general\/\" target=\"_blank\" rel=\"noopener\">General category.<\/a><\/span><\/strong><\/p>\n<\/blockquote>\n<p><span style=\"color: black;\"><a style=\"color: #ff9900;\" href=\"https:\/\/www.macleans.ca\/politics\/ottawa\/federal-election-2021-know-your-pollsters\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8220;#Federal election 2021: Know your pollsters&#8221; In his best-selling book Cracking the Quebec Code, Jean-Marc L\u00e9ger wrote\u00a0\u201cPolitical polls make for about 1% of all my revenues, but account to 99% of my problems.\u201d So, why do it then? For many market-research firms, political polling serves as an efficient way to promote their brand, attract customers&#8230;<\/p>\n","protected":false},"author":1,"featured_media":317371,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/www.macleans.ca\/wp-content\/uploads\/2021\/08\/CP22018167-766x431.jpg","fifu_image_alt":"","footnotes":""},"categories":[1],"tags":[68722,67806,109265,70468],"class_list":["post-317370","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-general","tag-338canada","tag-editors-picks","tag-federal-election-2021","tag-polls"],"_links":{"self":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/317370","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/comments?post=317370"}],"version-history":[{"count":0,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/posts\/317370\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media\/317371"}],"wp:attachment":[{"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/media?parent=317370"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/categories?post=317370"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/buradabiliyorum.com\/en\/wp-json\/wp\/v2\/tags?post=317370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}