It’s been no secret that polling is under fire. Heck, I’ve written about the topic a handful of times myself. Several peers have written in depth articles defending what we do. Still, the story continues to be debated in the public square. Clients still have questions that let me know they are concerned about today’s polling environment.
My point of view is that there hasn’t been a reduction in good polling. We live in a time when there is more good political polling than ever before. Pollsters who are good at what they do have access to tools and technology that help us hone in on the exact population that we intend to research, much more so now than in the past. We can meld different modes together to reach more of the population to make our samples very representative. The proliferation of great data and available analytics allow us to draw highly valuable insights into our samples and apply those to the population in the right ways.
So why is the polling industry taking it on the chin. Because the number of bad actors operating under the guise of survey research have increased too. Unrepresentative samples that are poorly conducted make their way to public release and muddy the waters when aggregated. Sometimes, when prolifically produced or hidden by beautiful presentations, these seem to give an impression of credibility.
There are ways to protect yourself as a polling consumer. Last week Jennifer Duffy of the Cook Report spoke about some of these flawed polling methods. Here are the points she made:
Beware Online Only
There is zero chance that an online-only poll will be representative enough to reflect a voter universe. If you are only looking at sentiment or trying to identify themes then perhaps this can be okay. When it comes to predicting a hard choice like how any individual voter will act in the voting booth on election day then that’s an entirely different matter. Online data collection is cheaper than phone, I get it. It’s the new thing, the cutting edge technology that everyone wants to say they use. If you want to utilize it, fine, but use it as a mixed mode piece in a project. Combining it together with phone collection is possible, but it has to be done just right for the sample to work accurately.
Too Long in the Field
Polls are a snapshot in time, just like the hard action of voting a predictive political poll measures. If a poll is in the field for weeks or heaven forbid, months, an individual could change their minds several times due to the environment they are in. Most polls need a few days in order to reach an adequate sample, but much past that then you are running a risk of the inclusion of bias inducing sample error in your results.
Here are a few more points to consider as an informed consumer of polling.
- Who Sponsored the Poll. Important because it can let you understand more about the purpose of the results. Some polls are not released to predict the outcome of a race. Their intent may be to influence the outcome of a race. That might mean the results are in no way indicative of what will actually happen.
- Who Responded to the Poll. Look at the demographic and geographic data in the crosstabs. You are looking to see if the sample is representative of the right population. If it’s not then the sample stands a very high chance of being invalid.
- Were Cell Phones Included. This is very similar to online only. Each piece of data collection (landline, cell, online) works to make a sample more representative. If cell phones are not included that can mean a sample is missing a large percentage of the population and the results will be skewed.
- What Questions Were Asked. The wording of questions is highly important to obtaining accurate results. Were leading questions asked before a head to head question? Did some individuals have their title or party affiliation added and others not? What was the order and context? There are several question traps that can cloud the ability to accurately predict an outcome.
Conclusion
In comics, movies, tv shows and the like, a character who can tell the future always has value. It’s a superpower just like flying or super strength. I say that less to pretend I’m a superhero, but more to speak to how people crave that ability. That’s one of the central reasons why polling is sought after and why predictive analytics have been all the rage during the past few cycles.
I believe that any campaign can benefit greatly from good polling. If you’re going to do it though, it’s only worthwhile if you do it right.
I enjoy talking about this stuff, ping me anytime. Find me here.