A new scoring approach introduced in 2009 to curb bias during the National Institutes of Health (NIH) Enhanced Peer Review process did not mitigate the gap in preliminary overall impact scores between black and white principal investigators (PIs) for the years 2014 to 2016, a new study shows. Black investigators, on average, received worse preliminary scores for all five criteria - Significance, Investigator(s), Innovation, Approach, and Environment - even after accounting for career stage, gender, type of degree, and scientific field. Elena Erosheva and colleagues also demonstrated that substantial funding gaps continue to exist between black and white applicants, with the award probability for black PIs only 55% of that for their white counterparts. Although the authors caution against making direct comparisons to work done before the NIH introduced scored criteria to increase transparency to its applicants, this finding aligns with those of a series of groundbreaking studies from Donna Ginther et al., which demonstrated large funding disparities for black PIs during the years 2000 to 2006. In the interim, the NIH introduced its Enhanced Peer Review process in 2009 to improve information and transparency for applicants, an adjustment that the researchers hypothesized should have reduced funding gaps between black and white PIs. This process requires each reviewer to provide whole-number scores (ranging between one and nine, with one being the best) based on the NIH's five criteria, which the reviewer then uses to derive one preliminary overall impact score. Reviewers are given some latitude to weigh their criterion scores as they see fit when they derive their preliminary overall impact score. The averages of these preliminary overall impact scores determine which applications are selected for discussion at Scientific Review Group meetings, after which reviewers assign the final overall impact scores that decide an application's fate. To evaluate whether there are differences in how preliminary criterion scores of black and white applicants are combined to produce preliminary overall impact scores, Erosheva et al. performed multilevel modeling on preliminary scores from 54,740 NIH grant applications. The results demonstrate that preliminary criterion scores fully account for racial disparities in preliminary overall impact scores, although they do not explain all of the observed variability. "Overall, we conclude that preliminary criterion scores absorb rather than mitigate racial disparities in preliminary overall impact scores," the authors say.
###
Journal
Science Advances