In new research, Markus Eberhardt, Giovanni Facchini, and Valeria Rueda delve into a unique database comprising 12,000 reference letters, which were written in support of more than 3,700 applicants applying for academic job positions in economics in the United Kingdom. Their analysis uncovers a pervasive disparity in the way male and female candidates are recommended. Specifically, the authors observe that women are frequently lauded for their hard work and determination, and at times less likely to be praised for their natural talent. They also show that such gender-based stereotyping hinders the progress of women economists.
Academia is facing increased scrutiny over gender imbalances, and this is especially true in economics. Recent empirical work has documented that in our discipline, the career pipeline for women is “leaky.” This means that women drop out of the profession at critical transitions, such as the jump from earning a PhD to an assistant professorship or before obtaining tenure. In a recent paper, we study the first step of the academic career of an economist, the junior “job market.” This is the stage at which the leak has grown the most in the past decade and that so far has not received much systematic attention in the literature.
The academic job market for economists is a distinctively structured and centralized institution. Each fall, universities release job postings, and aspiring candidates prepare a comprehensive “job market package.” This package includes one or more academic papers, a curriculum vitae, and a collection of reference letters written by scholars familiar with the candidate’s work. Candidates, letter writers, and hiring committees all interact through centralized platforms in this market. Since the same package is typically used for most job applications, the incremental cost of submitting an additional application is low. Candidates typically use the same reference letters for all applications, rather than tailor them to individual institutions. Consequently, research-intensive institutions receive large numbers of applications, and the sample considered in our study can be considered a broadly representative slice of the reference letters present in the market.
We obtained access to data comprising all applications for entry-level positions received by one research-intensive university in the U.K. between 2017-2021. Deploying natural language processing tools, we analyzed the text of 12,000 reference letters written in support of over 3,700 candidates. A standard letter covers a lengthy discussion of the candidate’s job market paper, some mention of their additional research, and their teaching and citizenship skills. The final section provides a summary assessment of the candidate’s academic abilities and recruitment prospects—this section is the focus of the main analysis carried out in the paper.
We use two complementary approaches to study the letters. We first build a model that predicts whether a letter was written in support of a woman or a man. Unsurprisingly, terms related to research have strong predictive power; certain fields such as family economics attract women. Less easily explained by the candidate’s academic choices is the fact that personality traits (“shy,” “pleasant”) and grindstone attributes (“driven,” “hardworking,” “determined”) also feature as strong predictors of letters written in support of women.
We then rely on a second model to assess whether the gender of the candidate predicts whether the letter refers to certain attributes. To categorize the attributes present in letters, we are informed by existing research on the topic proposing that five traits are usually highlighted in academic reference letters. These are ability traits, grindstone traits (e.g., “hardworking”), research terms, standout adjectives, and teaching and citizenship terms. We added a category that refers to the recruitment prospects of the candidate and validated the dictionaries we defined for each trait with a survey of a large sample of academic economists based in U.K. research-intensive universities.
Our results establish that the praise of hard work is significantly more likely to appear in letters written in support of women. This pattern is robust for candidates across all ranks of PhD-granting institutions (many of the candidates were international and had received their PhDs at schools outside of the U.K.), and independently of whether their letter writer is a male or a female researcher. The most striking effects are observed for letter writers who have less experience working with female students and those who are not the student’s main advisor and are therefore less likely to know them well. For these letter writers, we observe a sharp divergence in the qualities praised: they are largely and significantly more likely to praise hard work and less likely to praise natural ability. Finally, although men and women are equally likely to receive negative comments (“I recommend this person to any department outside the top-50”), women are less likely to receive outstandingly positive ones (“any department including top-5”).
Does praising hard work impact career prospects? After all, being keen and driven is a positive thing for an employee. However, it’s also expected of everyone in a competitive and highly skilled profession. More broadly, sociologists since the work of Pierre Bourdieu and Jean-Claude Passeron in the 1970s have pointed out that minorities are more often praised for their diligence rather than for their innate ability, and that such praise can be interpreted as a lack of natural talent. More recently, Chia-Jung Tsay and Mahzarin R. Banaji, focused on professional musicians and documented that recruiters in a controlled experimental environment systematically chose candidates that were praised on their natural ability compared to those praised on their “grit.” This favoring of “naturals” over “hard workers” is revealed despite these recruiters’ stated preference for hard work over natural ability.
We illustrate the consequences of language bias in recommendation letters by collecting placement information for all the applicants in our sample. These data show that for academic jobs, the praise of diligence and hard work is significantly associated with landing a job in a less prestigious institution—a penalty which is markedly higher for women. Our research sheds light on the role of implicit stereotyping in explaining the slower advancement of women as academic economists. Letter writers devote a lot of time in writing and polishing letters, and the overwhelming majority likely cares about their students and aims to place them as highly as possible. Therefore, it is unlikely that, on average, they are deliberately undermining female students by emphasizing less desirable attributes. Recent research has shown that unconscious biases can be addressed by providing the actors involved with evidence of the existence of such biases. By shedding light on these patterns, we hope this research will be a first step towards increasing awareness and reducing possible stereotyping in the job markets.
Articles represent the opinions of their writers, not necessarily those of the University of Chicago, the Booth School of Business, or its faculty.
