Not a conversation has gone by in the past several months that hasn’t devolved into “what’s wrong with polling” after someone finds out I work at a polling firm.
And there’s good reason for the general public – and even practitioners – to doubt the accuracy of political polls.
Let’s step into some minutia quickly about what makes polling work, which will explain why the old way (live telephone calls) and emergent ways (mostly opt-in online panels) of polling are broken. Gathering opinions from 600 people statewide can predict within a margin of error the opinions of the entire population ONLY if the 600 respondents are representative of the whole.
Back when a pollster could call 6,000 voters and get a 10% response rate on landlines and cell phones, it was easy to fill a representative sample. Ah, those were the days. However, telephone response rates now range anywhere from 1-3% and the most important population groups – minority voters, younger voters, and low-propensity voters – have response rates of <1%.
On top of that, respondents from hard-to-reach groups who answer an unknown telephone call and speak to a stranger for 20 minutes look different from those who do not respond (creating non-response bias). Pollsters then add weight to these non-responding voter groups so that, for example, the response from the one 25-year-old Asian female who is an Independent that occasionally votes in presidential elections is tripled to represent the three individuals actually needed from that subgroup to have a representative sample.
Weighting adds bias by stressing the final sample, which decreases the Effective Sample Size—our metric for a well-collected, representative poll. Many traditional pollsters are ok adding 20-30% total weighting to their samples to force them to be “representative.” This reduces a 600 sample survey down to either a 480 or 420 Effective Sample Size and increases the margin of error significantly.
Therefore, polls that aren’t representative during sample collection (getting people to answer the poll questions) only get worse when weighting is applied. All this is before the conversation of turnout projection, another key component of having accurate poll results.
Since this “quick” explanation of polling basics became five paragraphs, I’ll actually briefly explain the problem with online polls (what many public polling outlets use). People who take surveys online opt-in to participate; many are “professional” survey takers, taking dozens of incentivized surveys per day. This approach is non-probabilistic because not everyone in the target population – likely general election voters in the state of Texas per se – has the chance to participate in the poll like a telephone or multi-mode survey. One day online panels will be large and representative enough to take the full place of probabilistic polling, but we’re a few years away from that.
Blink your eyes a few times now. I realize this was a lot to consume and process. But now you understand all of the challenges with telephone-only polling and online-only polling. That’s why you feel that “polling is broken.”
The problem with traditional polling shops is that they have never been responsible for gathering the responses from voters; that’s always been something their phone center handled.
What has made Cygnal insanely accurate and insightful is that three years ago we recognized this problem and took over the responsibility of communicating with voters to get their opinions for our polls. We do this by using communication channels how real people use them.
Think about your smartphone, which we really shouldn’t even call a phone anymore. It’s a smart device, a multi-faceted computer in our pocket that happens to occasionally be talked on. Do you use your iPhone exclusively for talking by voice through a phone call? NO! So polls should not be using this mode exclusively to talk to voters to gather opinions.
Representative polls now incorporate communicating with voters by text message, email, apps, social media networks, and yes, even telephone calls. Eventually, it will be exclusively online, but we’re not there yet…give us a couple of years as we fully flesh out Cygnal 3.0 that we’re already building. 😎
It’s really just common sense if you think about it.
…I see you shaking your head in agreement. I am also anticipating your next question: “Why aren’t all pollsters doing this?” Well, because it’s hard, really hard. Conducting a multi-mode poll literally takes three times the amount of work compared to just writing a script, sending it to a call center, and having them send you back the completed toplines and crosstabs. It requires staffing your polling firm differently as you take over data prep, coding, sample collection, and reporting functions.
Several old-school shops are trying, but their approach is just taking the telephone calling approach to other modes of communication or simply outsourcing it to a vendor. It doesn’t work like that. There’s a reason we employ full-time software developers to build tools for our team to more efficiently conduct our accurate multi-mode polls and predictive analytics projects.
Ultimately, we believe you’ll continue to see public polling that is bad because doing it the right way is hard. It is more expensive than online panel polls but less expensive than live-telephone-only polls. Heck, change is hard. It’s uncomfortable. It’s trying…but it’s worth it.
That’s why Cygnal is so different from any other polling firm, and we’re proud of it.