In its early days, political opinion polling’s leading advocate, George Gallup, sold it as an essential tool for democracy. He believed polling made for better representation because it allowed politicians to take the people’s “pulse”.

But opinion polling didn’t so much enhance democracy as remake it.

Thanks to Gallup, polls have become so ubiquitous in modern-day politics that we’re now convinced they can accurately predict elections. (Even though Donald Trump’s surprising victory in the 2016 US presidential election suggests otherwise.) Gallup ran his first poll in the US in 1935 and in Australia in 1941.

Since then, opinion polling has changed every liberal democracy by turning politics into a contest between two sales teams trying to synthesise a product they believe voters want and diluting what was once the key role of politicians: to provide leadership.

Polls driving the news cycle

In Australia, this can be seen in the revolving door of prime ministers over the past decade. Polling isn’t the sole reason for this political instability, but it’s played an important part.

Obsessive poll-watching has become standard practice for politicians, as well as the journalists who cover them. This is partly because polls have become news stories in themselves, and not just at election times. A new poll is “news” because it provides the latest measure of the mood of the electorate, which is what everyone wants to know.

The weekly countdown of Malcolm Turnbull’s losses in the Newspoll is a case in point. Because Turnbull arbitrarily set a threshold of 30 Newspoll losses as his justification to challenge the leadership of Tony Abbott, the media fixated on the same arbitrary threshold during his time in office.

When Turnbull lost his 30th straight Newspoll, the media made it feel like a death knell.

n her parting shot to her party, Gillard made clear what she felt had contributed to its decline in leadership:

"…real thought has to be given to how to make any leadership contest one in which candidates have to articulate why they want to lead Labor and the nation. … The identification of the top new ideas – not just who is top of the opinion polls. "

As soon as Scott Morrison was picked to replace Turnbull, all eyes turned again to the polls to see how the electorate would respond.

In the latest Newspoll, the Coalition had closed the gap with Labor somewhat, but still trailed overall 46-54%. The Fairfax-Ipsos poll showed similar numbers.

So, not much has changed for the Liberals and it appears not much will – they’re stuck with Morrison now. Some are probably asking themselves now if the spill was worth it, particularly with so many marginal seats in play in the next election and the Coalition sitting on a one-seat majority.

The impact on decision-making

A less visible effect of polling has been the impact it’s had on conversations inside the major parties.

In some regards, policymaking is no longer based solely on a leader’s principles and what the party stands for. It’s about which policies are most likely to keep the party ahead in the opinion polls.

It’s becoming increasingly unlikely for the inner core of senior politicians who run the parties to push through a necessary, but unpopular, policy with the goal of changing the minds of voters who don’t agree with it.

Take Australia’s contentious asylum seeker policy, for instance. Following record numbers of boat arrivals in 2012, many polls were taken to gauge the public’s opinion on the Gillard Labor government’s handling of the issue.

The results showed a high degree of confusion. As many as one in five respondents reported uncertainty in a number of surveys. When that happens, a minor change in a poll’s wording can shift the results in major ways.

But those who wanted to turn back the boats were far more entrenched. In a 2012 survey by the Scanlon Foundation, 26% of respondents favoured “turning the boats back” as a solution to the crisis. Other polls showed that voters overwhelmingly blamed the government for the impasse.

There was an opportunity for our leaders to step in with a solution that would bring together the 74% of people who didn’t support a “turn back the boats” policy.

But faced with negative headlines and an unhappy electorate – only 6% of respondents in the Scanlon survey thought the government was doing a good job on asylum seeker policy – it was far more expedient for the government to take a hard line than to craft and sell a more nuanced approach that would address people’s concerns and provide a more humane outcome for asylum seekers.

Potential problems with polling

Another troubling aspect of polls is that the numbers are less real than they are made to look. Hard as they try, pollsters are increasingly having a harder time finding a representative sample of people to survey.

According to Cliff Zukin, the former president of the American Association for Public Opinion Research, election polling is nearing a crisis:

Two trends are driving the increasing unreliability of election and other polling in the United States: the growth of cellphones and the decline in people willing to answer surveys.

The Pew Research Centre, for example, reported that 36% of those called in the US would agree to be polled in 1997 and only 9% agreed in 2016.

Many pollsters believe that IVR (interactive voice response), or robopolling, is the future. This automated software allows pollsters to make a higher volume of calls to compensate for the higher numbers of
hang-ups. Robopolling is also much cheaper.

In Australia, Newspoll stopped surveying people by landline phones in 2015 and shifted to a mixed methodology of robopolling and online surveys. The new Newspoll was found to be less prone to random fluctuations, but appeared to lean a little to Labor, relative to other polls.

Ipsos still relies on live phone polling, both land lines and mobiles. While Ipsos’ polling results are generally well-regarded, some analysts have found them to underestimate Labor and overestimate the Greens.

Despite all these questions about the accuracy of polls in the mobile phone era, however, they did appear to provide an accurate prediction of the 2016 general election in Australia.

While this is perhaps reassuring, it will only continue to fuel their appeal. As journalist Gay Alcorn put it, Australia’s obsession with polling is not only dispiriting, but corrupting for our politics:

What’s sad is that we know it, but find it impossible to rise above it.