While on a panel in Las Vegas for the 2017 Reed Awards, we dove into the topic of the reliability of polling.
After the well-documented number of polling misses revealed in the November election, the polling industry was deemed unreliable and inaccurate. However, it should be noted that myself and other pollsters I hold in high regard didn’t see problems with the work we were putting out. Like news, polling has its own real and “fake” versions.
When I mention real polling, I’m talking about the polling that is meant for the true measurement of public opinion. Not the stuff where it’s about persuasion or suppression instead of unbiased research. Not the stuff that is put out there to cause coverage of a certain slant or the crazy free for all IVR-only polling that shows no regard for representative samples.
Most everyone I know in polling has had to take it on the chin in some way recently. That’s like getting sucker punched for wearing an Auburn shirt because some other idiot I don’t even know was wearing an Auburn shirt and said something stupid to an Alabama fan. Perhaps this will help set the record straight.
Not All Polling is Real
If you read a poll in a publication, please do not take it at face value. Some results that are published were fielded incorrectly. Heck, some results were even published with bad intentions.
Take time to be a good consumer. Read through the questionnaire and look for imbalances in the questions or the responses. Look at the sample results and try to determine the poll’s representativeness. Form relationships with legit pollsters and ask them for their input about results you see.
Find Legit Pollsters
Ask around your network. Chances are you will get some referrals. Do your own research and look at the work they may have made publically available. Look at who they may have worked with and let that help you evaluate credibility.
You can do a google search and find helpful academic articles that will educate you on good questions to ask pollsters in evaluating their level of expertise. Take the time to select good practitioners who care about doing survey research the right way.
Garbage In, Garbage Out
Polling is an art and a science. There are several things that have to happen correctly in order to field accurate results. Not all polls are equal, but several polls around the same races get aggregated and scored by a cottage industry of polling aggregators. It does not take too many bad surveys to put those numbers out of whack. These aggregators should not be taken at face value either. Look independently at the surveys they are using to inform their models.
Conclusion
The biggest difference you can make for yourself is to help put a good product on the field. A ton of great pollsters exist, as do a fair share of PINOs (Pollsters in Name Only). The fakers can try to make it hard on those of us striving to do things the right way, but if we are helping our industry professionals become better consumers, than we’ll all see how real polling is not broken.
I answer questions on this stuff frequently. If you want to know more, please ping me anytime. Find me here.