University researchers have carried out the largest systematic review and meta-analysis to date of how people's perceptions of their screen time compare with what they do in practice, finding estimates of usage were only accurate in about five per cent of studies.
The international team say this casts doubt on the validity of research on the impact of screen time on mental health, and its influences on government policy, as the vast majority rely on participants to estimate (self-report) how long they spend on digital devices, rather than logs of actual usage, or tracked time.
"For decades, researchers have relied on estimates of how we use various technologies to study how people use digital media and the potential outcomes this behaviour can lead to. Our findings suggest that much of this work may be on unstable footing," said lead researcher Dr Doug Parry at Stellenbosch University.
"The screen time discrepancies highlight that we simply do not know enough yet about the actual effects (both positive and negative) of our media use. Researchers, journalists, members of the public, and crucially policy makers need to question the quality of evidence when they consider research on media uses and effects. We can no longer simply take claims of harmful effects at face value."
The researchers also investigated whether questionnaires and scales addressing 'problematic' media use, such as excessive or so-called 'addictive' media use, were suitable substitutes for logged usage. They found an even smaller association with usage logs for these measures.
Published in Nature Human Behaviour, the research identified every existing study that compares logged or tracked media use measures with equivalent self-reports. They screened over 12,000 articles for inclusion and found 47 studies that included both types of measures. From here they were able to identify and extract 106 comparisons, based on 50,000 individuals, to address the question of how closely self-report estimates relate to logs of actual usage.
"These highly flawed studies are over-inflating the relationships between digital media use and typically negative outcomes, such as mental health symptoms and cognitive impairments, which of course explains the pervading view that smartphones among other technologies are bad for us," said Dr Brit Davidson from the University of Bath's School of Management.
"Media and technology use takes the blame for everything from increases in teenage depression and suicide to higher incidence of Attention Deficit Hyperactivity Disorder (ADHD) and violence. If we want to properly investigate harms, we must first tackle assumptions about screen time and disentangle how people are actually using their phones or other technologies of interest.
"Importantly, these questionable studies are also being used to influence policy. The UK and Canada both have forms of screen time guidelines based on poorly conducted research, which is clearly worrying and hard to reverse."
The research team also included Dr Craig Sewall at The University of Pittsburgh; Dr Jacob Fisher at the University of Illinois Urbana-Champaign; Hannah Mieczkowski at Stanford University; and Dr Daniel Quintana at the University of Oslo.
The researchers hope that their study will lead to a shift in measurement practices regarding technology, alongside starting to correct the narrative regarding technology and society. They say that only by better understanding what people actually do with their technologies, can we start to genuinely understand the impact of them on people and society.
A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use is published by Nature Human Behaviour:
###
The pre-print paper is available from the press office, details below
The published paper will be available at: https://dx.doi.org/10.1038/s41562-021-01117-5 To contact the press office please email Alison Jones at a.jones4@bath.ac.uk or call 07966 341322.
University of Bath
The University of Bath is one of the UK's leading universities both in terms of research and our reputation for excellence in teaching, learning and graduate prospects. The University is rated Gold in the Teaching Excellence Framework (TEF), the Government's assessment of teaching quality in universities, meaning its teaching is of the highest quality in the UK. In the Research Excellence Framework (REF) 2014 research assessment 87 per cent of our research was defined as 'world-leading' or 'internationally excellent'.
From developing fuel efficient cars of the future, to identifying infectious diseases more quickly, or working to improve the lives of female farmers in West Africa, research from Bath is making a difference around the world.
Find out more: http://www.bath.ac.uk/research/. Well established as a nurturing environment for enterprising minds, Bath is ranked highly in all national league tables. We are ranked 6th in the UK by The Guardian University Guide 2021, and 9th in both The Times & Sunday Times Good University Guide 2021 and the Complete University Guide 2021. Our sports offering was rated as being in the world's top 10 in the QS World University Rankings by Subject in 2021.
Journal
Nature Human Behaviour