Anonymizing research funding applications could reduce ‘prestige privilege’

From Science

Reading Time: 4 minutes

20/04/2024

For research funders seeking to minimize bias in their selection process, removing applicants’ institutional affiliations from their submissions could help address a common disparity: disproportionate funding going to those at the most prestigious places. That’s the finding from researchers at the Arnold and Mabel Beckman Foundation, who reported last month in eLife that anonymized applications partially reduced the effects of reputational bias and evened the playing field for early-career faculty at lesser known institutions applying for their Beckman Young Investigator (BYI) awards.

The findings illustrate how some applicants “enjoy a leg up in grant writing, based on the name of their institution alone,” says Daniel Larremore, a computer scientist at the University of Colorado (CU) Boulder who studies inequity in academia; a 2022 study he co-authored found that 80% of U.S.-trained faculty come from just 20% of universities. He adds that the new study also provides an excellent strategy for mitigating this bias, which funders can use to make their review processes fairer.

The study adds to a growing body of knowledge about what anonymizing funding applications can—and can’t—do. The National Institutes of Health (NIH), for example, experimented with concealing information about applicants’ identities from peer reviewers as part of an effort to understand why Black applicants are 35% less likely than white researchers to receive grants. The results were mixed—Black applicants’ scores did not improve, but those for white researchers decreased—and some experts were skeptical about whether the reviewers truly didn’t know the identities of the applicants. More recently, NIH announced it would also be trying to address reputational bias—as of a few years ago, about 70% of all grants went to a mere 10% of all NIH-funded institutions—but the agency aims to do so not with blinding, but by adjusting the review criteria. And the Bill & Melinda Gates Foundation found that even with blind review, women applicants received lower reviewer scores, which the researchers attributed to women using narrower, more topic-specific language than men when describing their research. The new study, in contrast, seems to definitively show that blinding—at least in some cases—can have a tangible impact on reducing bias.

Anne Hultgren, who serves as an executive director at the Beckman foundation and was the lead author of the new eLife study, says the foundation’s BYI awards are intended to be the “first spark” for up-and-coming scientists who “who don’t yet have the clout to earn federal research dollars.” But when Hultgren and her colleagues reviewed the distribution of award applicants who were being invited to submit full proposals, they noticed “a lot of names coming up over and over” and began to wonder whether some researchers were enjoying an unfair advantage. In an attempt to balance the scales, the foundation decided in 2020 to blind the first stage of the review process for the awards by asking each applicant to omit certain identifying information—specifically name, gender, and institutional affiliation—in their initial Letter of Intent, which is reviewed by a panel of tenured researchers.

To find out whether the change had any effect, Hultgren and her colleagues compared data from the first 4 years of the new policy to the last 4 years of the old one, spanning more than 2000 total applications. They didn’t uncover any evidence of gender bias in any of the years, but they did notice a shift in so-called “prestige privilege.” Before blinding, researchers affiliated with institutions regarded as more prestigious—as ranked by the study’s authors, based on strength in the foundation’s focus areas, chemistry and the life sciences—were more likely to attract attention from reviewers and tended to progress further in the application process, with 75% of the resulting awards going to applicants at 25 top institutions. After institutional information was anonymized, this trend weakened—only 45% of awards went to applicants in the elite group—but didn’t disappear.

Donna Ginther, a labor economist at the University of Kansas who co-authored the study first identifying the NIH racial funding gap in 2011, says several factors may have contributed to this persistent disparity. More prestigious institutions tend to receive more funding, have better facilities, and are able to hire “more capable faculty.” Blinding can help alleviate some bias, but it cannot unravel these deeper, structural issues.

Despite these hurdles, anonymization may still be a step in the right direction, says Aaron Clauset, a computer scientist at CU who co-authored the hiring study with Larremore. Going forward, both federal and private research funders should think carefully about how to design their application review policies. “Prestige rules everything around us in academia,” he says. Concealing that prestige from reviewers could shift the focus away from a scientist’s reputation and back to the “quality of ideas.”

First published in Science on the 18th April – read the original and more great content at doi: 10.1126/science.z3v43yx