Public Release: 

Excluding individuals who only have a mobile phone from pre-election polls affects the results

Elhuyar Fundazioa

This news release is available in Spanish.

"Excluding the population who only have a mobile phone from pre-election telephone polls leads to significant biases in assessing voter intention: the vote for parties situated on the right of the ideological spectrum is overestimated and that for the ones on the left of it is underestimated." This is the main conclusion of the PhD thesis of Sara Pasadas del Amo, written up at the NUP/UPNA-Public University of Navarre; in it she analysed the effect of the lack of coverage or failure to cover this population on the level of accuracy of pre-election telephone polls.

The author of this piece of research stresses that the results "also show that the lack of coverage of this population is highly structured around significant variables of social differentiation, so it is to be expected that this problem will also lead to biases in the measuring of other subjects of interest for social and political research in our country."

Her PhD thesis is entitled: "Población 'solo móvil' y precisión de las encuestas preelectorales basadas en el modo de administración telefónico. El caso de las elecciones andaluzas de 2012" ('Mobile only' population and accuracy of pre-election polls based on the telephone administration mode. The case of the Andalusian elections of 2012). Her supervisor was Prof Vidal Díaz de Rada Igúzquiza of the NUP/UPNA's Department of Sociology, and she obtained a distinction cum laude.

The work has its starting point in the case of the study provided by the regional Andalusian elections of 2012 in which the predictions of all the pre-election polls were wrong. "These polls predicted a distance of the PP [conservative] ahead of the PSOE [socialist] that ranged between nine and fifteen points and which in the end was reduced to little more than one point." At the same time, in order to reinforce the reliability and validity of the research, the author repeated part of the analysis for the Andalusian vote in the 2011 general elections and for the national vote in the European elections of 2014.

Threat to credibility

As Sara Pasadas del Amo pointed out, the lack of accuracy of election polls "is a complex matter that can only be explained on the basis of a conjunction of various factors which have been dealt with at length in the sphere of political science and poll methodology".

The reasons behind this lack of accuracy can be grouped into two categories: factors relating to the behaviour of the people interviewed and the electors, some of whom state one thing in the polls and do something different on polling day; and the factors relating to the technical and methodological design of the polls.

The thesis focusses on the problem caused by the fact that when predicting the election results, telephone polls are only done over landline telephones, "thus excluding between 20% and 30% of the Spanish population who only have mobile phones". This is one of the main reasons why there are increasingly fewer people who have any chance of being contacted to participate in a survey.

In the case of phone interviews, bearing in mind the way they are administered in nearly all the pre-election polls conducted in Spain, it is estimated that nearly half the population have no chance of being part of the samples, either because they do not have a landline phone at home or because their numbers do not appear in the telephone directories that many of these surveys use as the sampling frame. "To this segment of the population that cannot be located by phone, which in itself is high, one would have to add the population that refuses to participate in polls of this type, and who, if they agree to respond, refrain from the questions about voting intention."

Beyond the methodological consequences with respect to the quality of the results, the author of the research considers that this lack of coverage "may go as far as signifying a considerable threat for the credibility of the poll as a research technique and, by extension, for the trust of the general public in the organisations that carry them out in the sense that this entails overlooking the opinions of a significant sector of the population."

The results, according to Sara Pasadas, highlight the importance of "incorporating whenever possible the mobile lines into the phone polls that are carried out in Spain and opting, in any case, for technical and methodological designs that give priority to data quality over other aspects like cost and obtaining the results quickly".

###

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.