Ahead of Wimbledon fortnight (23 June to 6 July), researchers from Cardiff University are advising that sports decision aids such as the Hawk-Eye system should come with a 'health' warning attached.
Hawk-Eye, and similar officiating tools which are used to supplement or replace decision-making by umpires and referees, are becoming an increasingly common part of televised sports coverage. But the new technology has also attracted its critics.
They include Wimbledon champion Roger Federer, who during his Final in 2007 against Nadal, claimed the technology was flawed. Nadal hit a ball which appeared to television viewers, to the umpire, and to Federer as impacting well behind the baseline, but Hawk-Eye called it in. Federer appealed to the umpire but the umpire accepted Hawk-Eye's judgment.
New research from Cardiff School of Social Sciences draws on the claims made by Hawk-Eye Innovations' on its website about the disputed line call. The machine reported that the ball nicked the baseline by 1mm. However, Hawk-Eye Innovations also report that the average error of the machine is 3.6mm. If the Cardiff analysis is correct, the errors can be even larger than 3.6mm on some occasions. The International Tennis Federation, which tests the machines for use, would accept that Hawk-Eye had passed its test if it called the ball in by 1mm while the true position was out by 5mm.
The Cardiff paper 'You cannot be serious! Public Understanding of Technology with special reference to `Hawk-Eye', due to be published in the journal Public Understanding of Science in July challenges whether aids such as the Hawk-Eye system can actually always be right.
Led by Professor Harry Collins and Dr Robert Evans, the team argues that such devices could cause viewers to overestimate the ability of any technological devices to resolve disagreement among humans. It also suggests that a more detailed understanding of how the device works could play a vital role in public education the benefits of which could spread to all technological decision making in the public domain.
Professor Collins said: "We used Hawk-Eye as the principal illustrative example as it is the most well-known of the commercial systems and is currently being used to make decisions in major tennis competitions such as Wimbledon. Technologies such as Hawk-Eye are meant to relegate line call controversies to the past, however, our analysis has shown that Hawk-Eye does not always get it right and should not be relied on as the definitive decision maker."
The paper puts forward the Automated Decision Principle. This states that automated sports decision aids should not correct but should reproduce human systematic errors -- the typical errors made by human judges and viewers such as calling a tennis ball `out' when it looks out to everyone even if the electronics suggest it might just have been in. At the same time, the devices should be used as they are now to correct or reduce human random errors, which come from lapses of concentration, an obscured view or very fast action - but the fact that the machine can also make mistakes should always be clear. If adopted, this Principle, would involve significant changes in the way devices such as Hawk-Eye are currently used in sport.
Professor Collins says: "To avoid any chance of misleading the public we believe that Hawk-Eye's conclusion should be accompanied by statements and/or displays of the size of the possible errors, as is normal in science. This would ensure that the public is much better informed as to the limits and possibilities of technology.
"Although such systems have not have been designed for this purpose, they could actually become a useful way to educate the public of the limits and possibilities of technology and serve to ask questions about the degree of certainty the public can actually attribute to scientific measurement and technological tools.
"What's more, even perhaps create a new role for sports commentators whereby they would interpret and explain levels of uncertainty for the viewer."
The analysis also concludes that Hawk-Eye might be in danger of unnecessarily changing the traditional nature of certain games because it does not take account of traditional areas of human systematic error when making judgements.
These systematic errors are an integral part of the sport. They include giving the benefit of the doubt to batters in the case of lbw or judging tennis balls to be out when skidding gives the appearance of their being out even if they have just clipped the line.
Notes to Editors
For more information about the research contact:
Professor Harry Collins
Director the Centre for the Study of Knowledge Expertise and Science
Cardiff School of Social Sciences
T: 02920 316603/02920 874 047
About the Research
An online copy of the study, 'You cannot be serious! Public Understanding of Technology with special reference to `Hawk-Eye' Public Understanding of Science, 17, 3, 283-308 can be accessed at:
A pre-print is available from the Cardiff University Public Relations and Communications Division or from www.cf.ac.uk/socsi/expertise.
About Cardiff School of Social Sciences
The School of Social Sciences encompasses teaching and research in social studies and education. Within social studies, the School has research interests in five main areas: criminology and criminal justice; health and medicine, knowledge and social change; modernity (time, risk, environment); and social welfare systems. Recent research includes investigation of drug use, white collar fraud, child welfare, domestic violence and European environmental policies. Many of the staff are international experts in their fields.
The School's education programmes are aimed at students who are interested in the psychological, social and political dimensions of education, as well as those who are vocationally-orientated. Education-related degrees are recognised by the British Psychological Society for the Graduate Basis for Registration, which provides the basis for future training in psychology-related professions. At a postgraduate level, it offers a range of research and taught Masters schemes in education, in addition to the PGCE for teaching in Further Education.
The School's research contributions in education management and policy; psychology of education and special educational needs; and post-compulsory education and training are internationally-recognised and are the foundation of its teaching programmes. In the independent government assessment of teaching quality, the School was awarded the highest, "Excellent" rating. In the most recent Research Assessment Exercise, the School was awarded a Grade 5 Star rating for Education and a Grade 5 for Sociology, indicating research of international excellence.