Laurie Schintler, Associate Professor, Schar School of Policy and Government, is set to receive funding for the project: "Perceptions of Efficiency and Bias in Peer Review: Algorithmic versus Human Decision Making."
This project is a collaboration between the Schar School of Policy and Government (GMU) and the Department of Sociology and Anthropology (CHSS-GMU). The investigators on the team include:
PI Laurie A. Schintler;
Co-PI Connie L. McNeely, Professor, Schar School of Policy and Government; and
Co-PI James Witte, Professor, Sociology, Sociology and Anthropology; Director, Institute for Immigration Research
This project will develop improved methods and concepts to guide the development and application of artificial intelligence technologies that could be used in peer review processes for evaluating scientific publishing and funding outcomes. The research team will seek to compare perceptions of peer review decisions assisted by algorithms to those made by humans.
The focus of the research is on the ethics and value of using algorithms in peer review. On the one hand, algorithmic peer review serves an instrumental purpose, purportedly offering the ability to make more efficient decisions. On the other hand, algorithms can produce biased and discriminatory decisions, which can raise ethical concerns about their use.
The research team will carry out a factorial survey based on experiments in which research participants are presented with vignettes regarding human and algorithmic peer review decision-making. The researchers will ask participants to assess the legitimacy of each scenario in light of bias and efficiency. The team will employ various techniques, principally multi-level econometric methods, to analyze data drawn from the survey and other sources.
The project will offer insights on measuring both pragmatic and moral legitimacy as they pertain to peer review. The project will also provide an adaptable survey tool and approach for gauging perceptions about algorithmic versus human peer review decisions and other scholarly communication activities.
Regarding the importance of the project, Schintler said, "This study will expand knowledge and perceptions of the benefits and downsides of using artificial intelligence in peer review and how science as a human and machine endeavor should be conducted."
Schintler will receive $399,922 from the National Science Foundation for this project. Funding will begin in Sept. 2023 and will end in late Aug. 2025.
###
About George Mason University
George Mason University is Virginia's largest public research university. Located near Washington, D.C., Mason enrolls 38,000 students from 130 countries and all 50 states. Mason has grown rapidly over the last half-century and is recognized for its innovation and entrepreneurship, remarkable diversity and commitment to accessibility. Learn more at http://www.gmu.edu.