Gender trends in computer science authorship

TL;DR

This paper analyzes 11.8 million computer science papers up to 2019, revealing a persistent gender gap in authorship that is unlikely to close this century without intervention, contrasting with faster progress in other fields.

Contribution

It provides a large-scale, up-to-date analysis of gender trends in computer science authorship and collaboration, highlighting the slow closing of the gender gap over 50 years.

Findings

Gender gap in authorship persists and is unlikely to close this century.

Cross-gender collaborations are slowly increasing over time.

Parity in authorship has been reached in fields like Medicine and Sociology.

Abstract

A large-scale, up-to-date analysis of Computer Science literature (11.8M papers through 2019) reveals that, if trends from the last 50 years continue, parity between the number of male and female authors will not be reached in this century. In contrast, parity is projected to be reached within two to three decades or may have already been reached in other fields of study like Medicine or Sociology. Our analysis of collaboration trends in Computer Science reveals shifts in the size of the collaboration gap between authors of different perceived genders. The gap is persistent but shrinking, corresponding to a slow increase in the rate of cross-gender collaborations over time. Together, these trends describe a persistent gender gap in the authorship of Computer Science literature that may not close without systematic intervention.

Tables

Table 1. Table 1. Corpus statistics for different fields of study.

Field of Study	Total papers	Total author- paper units	Average authors per paper
Art	5.3M	7.4M	1.4
Biology	15.1M	55.2M	3.7
Business	3.7M	5.8M	1.6
Chemistry	14.7M	48.6M	3.3
Computer Science	11.8M	27.3M	2.3
Economics	3.8M	6.4M	1.7
Engineering	10.1M	20.9M	2.1
Environmental Science	2.0M	4.6M	2.3
Geography	4.0M	7.3M	1.8
Geology	3.2M	8.4M	2.6
History	6.0M	8.2M	1.4
Materials Science	7.4M	21.7M	2.9
Mathematics	5.5M	10.9M	2.0
Medicine	32.4M	111.9M	3.4
Philosophy	2.8M	3.9M	1.4
Physics	7.8M	31.0M	4.0
Political Science	4.9M	6.8M	1.4
Psychology	7.0M	14.7M	2.1
Sociology	4.6M	6.3M	1.4
Total	152.1M	407.2M	2.7

Table 2. Table 2. RMSE of different curve fits for the proportion of female authors in each field of study since 1970.

Field of Study	RMSE (linear)	RMSE (exponential)	RMSE (logistic)
Art	0.14	0.11	0.10
Biology	0.09	0.07	0.02
Business	0.09	0.08	0.05
Chemistry	0.09	0.05	0.03
Computer Science	0.10	0.08	0.04
Economics	0.07	0.06	0.03
Engineering	0.13	0.11	0.07
Environmental Science	0.10	0.09	0.06
Geography	0.10	0.08	0.07
Geology	0.05	0.03	0.03
History	0.08	0.09	0.07
Materials Science	0.11	0.06	0.03
Mathematics	0.06	0.04	0.02
Medicine	0.14	0.11	0.03
Philosophy	0.09	0.05	0.04
Physics	0.06	0.03	0.02
Political Science	0.08	0.09	0.07
Psychology	0.07	0.11	0.05
Sociology	0.03	0.12	0.02

Equations

$${\varphi }_p(B)(1-B^d)y_t=c+{\theta }_q(B){\epsilon }_t$$

$$p_{mm}=m_1{m}_2;p_{mf}=m_1{f}_2+f_1{m}_2;p_{ff}=f_1{f}_2$$

Full text

gender, scientific authorship, authorship statistics, gender gap, bibliometrics

G. Stanovsky: Work done while at the Allen Institute for Artificial Intelligence and the University of Washington.

††doi: 10.1145/3430803††ccs: Social and professional topics Industry statistics††ccs: Social and professional topics Gender††ccs: General and reference General literature

1. Introduction

This paper presents a large-scale automated analysis of gender trends in the authorship of Computer Science literature. Specifically, we aim to address the following questions:

• •

How is gender balance among authors changing over time?

• •

When might gender parity be reached among authors?

• •

How is gender associated with co-authorship?

• •

How does Computer Science compare to other fields of study in gender representation among authors?

We answer these questions by performing an automated study of literature metadata from scientific conferences and journals, using data from the Semantic Scholar academic search engine.111https://www.semanticscholar.org/ Our study incorporates metadata from 11.8M Computer Science publications. To provide a basis for comparison, we also analyze more than 140M papers from other fields of study. We attempt to provide an overview of the relationship between gender and authorship in Computer Science, both throughout the history of the field, as well as in relation to other fields of study. Our results demonstrate that although progress has been made, there is still a significant gap in gender representation among Computer Science authors. Continued delay in addressing the gender gap may perpetuate imbalances for generations to come.

2. Data

Our analysis was performed over the Semantic Scholar literature corpus (Ammar et al., 2018). The corpus contains publications from between 1940 and the end of November 2019, and associated metadata such as title, abstract, authors, publication venue, and year of publication. Metadata in Semantic Scholar are derived from academic publishers, as well as scientific repositories like arXiv, DBLP, and PubMed. We use the 19 fields of study defined by Microsoft Academic (Shen et al., 2018), which are integrated with Semantic Scholar data. Table 1 shows the distribution of papers used in our analysis by field of study.

The author list is extracted from all publications and compiled into a list of first names. We use Gender API222https://gender-api.com/ to perform gender lookup for each name. Gender API is a large online database of known name-gender relationships derived by linking publicly available governmental data with social media profiles in various countries. For each name, Gender API outputs the predicted binary gender (female or male), along with the accuracy associated with the prediction and the number of samples used to arrive at that determination. We exclude authors for whom first names are missing, and for whom only first initials are available. We also filter out first names that occur less than 10 times in our overall corpus, to reduce the number of API calls to manageable numbers.

Because many names are ambiguous with respect to gender, we use the accuracy returned by Gender API to represent the gender of each author as a distribution over male and female probabilities. For example, Gender API estimates the first name Matthew to be male with an accuracy score of 100, the maximum. The name Taylor, however, is estimated to be female but only receives an accuracy score of 55. These accuracies are used to generate two probabilities for each name, $(m,f)$, where $m$ is the probability of the associated author being perceived as male, and $f$ is the probability of the associated author being perceived as female, where $m+f=1$. In this example, each author named Matthew will be represented with the probability tuple $(1.0,0.0)$, and each author named Taylor will be represented as $(0.45,0.55)$.

We acknowledge that gender identity is fluid and non-binary. However, for the sake of this large-scale study–we adopt a simplified view of gender as a probability distribution over two genders, relying on first names as a proxy for the author’s perceived gender (as opposed to self-reported gender). We use Gender API’s results as an estimation of authors’ perceived binary gender, and use these estimates to generalize over our corpus. We are not making claims about any author’s true self-reported gender.

3. Analyses

We perform two types of analysis on this data. First, we analyze publication trends, examining the number and proportion of female authors over time (§3.1). To identify when gender parity may be reached, we project the proportion of female authors based on trends from the last 50 years (since 1970). In this paper, we define parity as the proportion of female authors falling within 10% of 0.5, within the range of 0.45-0.55. We also study trends in co-authorship behavior as reflected in our data (§3.2).

3.1. Authorship analysis

Most papers are authored by more than one individual. For the purposes of our analysis, each author-paper pair is treated as one unit. A paper with a single author yields one author-paper unit; a paper with three authors yields three author-paper units and so on. In Computer Science , the average number of authors is approximately 2.3 per paper. However, average authors per paper have increased from approximately 1.5 per paper in 1970 to approximately 3.0 in the past several years, which reflects patterns observed by other researchers (Fernandes and Monteiro, 2016). Appendix B provides further discussion of this shift in relation to concurrent increases in author count in other fields.

The proportion of female authors over time is used to project the trend towards gender parity. The number of female authors in a given year is computed as the sum of probabilities $f$ over the author-paper units of that year, and the number of male authors is correspondingly generated as the sum of probabilities $m$. The proportion of female authors for each year $F_{t}$ is computed as the number of female author-paper units divided by the total number of author-paper units for the corresponding year. We compute projections by performing an autoregressive integrated moving average (ARIMA) analysis, a widely used and established method for creating time series forecasting models (Box et al., 1994). ARIMA is an autoregressive forecasting technique: which means it uses historical values in a time series to predict current and future values. We use the auto ARIMA function in the R ‘forecast’ package (Hyndman and Khandakar, 2008), which automates the selection of ARIMA model order, with a preference for simple models with lower order.

We assume that the growth in female author proportion observes logistic behavior. The proportion of female authors is necessarily constrained between 0 and 1, and logistic growth assumes that a stable equilibrium will eventually be reached. We tested other fit functions (linear and exponential; see Appendix C for details), but found them to be less suitable; the root-mean-squared-error (RMSE) of the logistic fit is lower than that of these other curve types when fitting to the growth curves of each field of study.

To perform the fit, we first apply $\sigma_{\alpha}^{-1}$, the inverse of the $\alpha$-scaled sigmoid (or logit) function $\sigma_{\alpha}(x)=\alpha/(1+\exp(-x))$, to map the gender proportion into the real line so that the data is more amenable to linear approximation. We call $\alpha$ the expected equilibrium proportion parameter. This transform generates $y_{t}=\sigma_{\alpha}^{-1}(F_{t})$, where $F_{t}$ is the proportion of female authors per year. We then fit a non-seasonal ARIMA model with parameters $p$, $d$, and $q$ for the transformed process $y_{t}$ represented by the following equation:

[TABLE]

where $B$ is the backshift operator, which shifts by one to the previous time point, and $\varepsilon_{t}$ is zero-centered, normally distributed noise (Hyndman and Khandakar, 2008).

Finally, we obtain the forecast in the original domain using a sigmoid transform over the projected values, applying $\sigma_{\alpha}$ to $y_{t}$ for $t>2019$. We sample $\alpha$ from the range $[0.3,1.0]$ so that $\sigma_{\alpha}$ has minimum and maximum values of 0 and $\alpha$ respectively. This constrains the projected values to be between 0 and some expected equilibrium proportion defined by $\alpha$. The 80% and 95% confidence intervals of the prediction are computed from averaging the projection confidence over 10000 iterations of model fitting.

The range for $\alpha$ is defined based on the space of likely equilibrium proportions, as estimated based on trends observed in various fields of study (see Figure 4). Note that $\alpha$ represents the proportion of female authors we expect in the long run. An equilibrium proportion of 0.5 indicates that we expect the authorship makeup to eventually stabilize at around 50% men and 50% women. An equilibrium proportion of 0.9 indicates that we expect the authorship makeup to eventually stabilize at around 10% men and 90% women. As is further elaborated in §4.2, we perform a sensitivity analysis to determine the effect of the selected $\alpha$ parameter on the year in which parity is expected to be reached.

3.2. Co-authorship analysis

Co-authorship is computed for each unique pair of author-paper pairs for each paper. If a paper has $n$ authors, $n\choose 2$ co-author pairs are generated. Given one co-author pair $(n_{1},n_{2})$ and associated gender probabilities $n_{1}\rightarrow(m_{1},f_{1})$ and $n_{2}\rightarrow(m_{2},f_{2})$, we compute three probabilities, $p_{mm}$, $p_{mf}$, and $p_{ff}$, corresponding to the gender combinations, i.e., between two male authors, a male and a female author, and two female authors respectively:

[TABLE]

where $p_{mm}+p_{mf}+p_{ff}=1$. The numbers of male-male, male-female, and female-female co-author pairs for each year are computed by summing over the above probabilities over all co-authorship pairs of that year.

We then assess the number of same-gender and different-gender collaborations over time. The results are measured as a deviation from the expected, where the expected co-authorships are determined by sampling from the numbers of female and male authors active in a given year, assuming the same number of collaborations per year as observed in our data. The total number of extra or missing collaborations is computed as the difference between the observed counts of each type of collaboration and the expected value. To show rates of change, we also compute the ratio between observed and expected collaborations (O/E) of each type.

4. Results

In the following section, we discuss the main findings of our study.

4.1. Gender API results

The 152.1M papers in our corpus resulted in 407.2M author-paper units. Of these author units, 14.5M lack first names, 110.0M have only a first initial, and 5.7M have a first name that occurs less than 10 times in the corpus. These author units are removed from further analysis. The remaining 277.0M author units are associated with 521K unique first names. We query these 521K names in Gender API, and acquire gender information for 351K; 170K names have insufficient information and are excluded from analysis. Of the 11.8M papers in Computer Science and the 27.3M author-paper units therein, 24.1M authors have valid first names, and 16.9M author-paper units (61.8%) resulted in associated gender information, which is higher coverage compared to authors in other fields (we acquire gender information for approximately 50.4% of authors across all fields).

4.2. Gender trends among authors

Figure 1 shows that the overall author count in Computer Science has increased substantially over the last several decades, as the field has experienced significant growth. The total number of author-paper units in 2018 is above 1.2M. The proportion of female authors has also increased during this time.

Figure 2 shows the projected proportion of female authors in Computer Science. The projected growth in female author proportion is computed using ARIMA. We assume logistic growth, and sample the $\alpha$ parameter for equilibrium proportion from the range $[0.3,1.0]$. We report an average projection computed over 10000 samples. Residuals of the ARIMA fit line over the logit-transformed data appear normally distributed, and are not significant under the Shapiro-Wilk Normality Test (Shapiro and Wilk, 1965). The proportion of female authors in Computer Science is predicted to reach 0.45 around 2124, more than 100 years from now. The upper bound of the 95% CI reaches 0.45 in 2065, and the lower bound of the 95% CI reaches 0.45 beyond the range of our projection. Appendix A provides further discussion on model choice and the sensitivity of ARIMA projections to the choice of the equilibrium parameter.

We also make the somewhat concerning observation that the rate of growth in female author proportion has slowed in recent years, visible in Figures 2 and 4. Our projection makes the optimistic assumption that the proportion will continue to grow towards or beyond parity, but the data may suggest otherwise. It remains to be seen whether a new trend is emerging that exhibits not an increase, but rather a leveling off or decrease in the proportion of female authors.

4.3. Association of gender and co-authorship

The numbers of same- (male-male or female-female) and cross-gender (male-female) co-authorships in Computer Science are computed for each year. Figure 3 shows the difference between the number of observed and expected collaborations of each type since 1990.333We show collaboration counts after 1990 because there is higher data volume in this period of time. In this time period, there are more same-gender co-authorships than would be expected, and fewer cross-gender co-authorships than would be expected. In recent years, around 50000 cross-gender co-authorships per year were missing when compared to expected numbers.

The observed to expected ratio shows both optimistic and pessimistic collaboration trends. Although both men and women are more likely to co-author with authors of their own gender (positive O/E), the degree of same-gender bias is declining among female authors but potentially increasing among male authors. At the same time, the cross-gender collaboration gap (O/E $<$ 1.0) is still rather large, such that in recent years, only around 90% of expected cross-gender collaborations are observed. In other words, although there are more opportunities for cross-gender collaboration in recent years (due to an increase in the number of female scientists working in the field), the observed number of cross-gender collaborations is still below what would be expected. Optimistically, these trends may be shifting in the recent past, with numbers from the last three years showing a shift towards more cross-gender co-authorship; although it is too early to say whether this tendency will preserve itself in the future.

4.4. Comparison of CS with other fields of study

Figure 4 shows the the proportion of female authors in 19 fields of study over the last 80 years. Computer Science is among the fields with the lowest female representation in recent years, despite having relatively higher female representation in the middle of the 20th century.

5. Discussion

Our analysis of the Computer Science literature reveals persistent patterns of inequality in gender and academic authorship. Although gender balance among authors is improving, progress is slower than we had hoped.

5.1. Limitations

Inferring gender from first names is imperfect, and all gender-inference tools are subject to biases. Several studies have described and measured the differences between these services (Karimi et al., 2016; Santamaría and Mihaljević, 2018). Based on results in Santamaría and Mihaljević (2018), Gender API has the lowest overall error rate but was slightly biased toward under-representation of females in their evaluation, in other words, the number of women estimated may be slightly lower than in reality. However, this bias may be offset by our sampling bias, since the population of CS authors is unlikely to be an unbiased sample of the general population, or the population whose names were used to construct the database behind Gender API. We attempt to mitigate some of these biases by treating the perceived gender as a probability distribution. One way to compute a more precise estimate is to weight the probabilities assigned by Gender API to each name using the prior probabilities of being a female or male CS author; this would likely produce a more pessimistic projection.

The proportion of authors with high uncertainty in Gender API results has also grown in our corpus over time. The average confidence of our gender predictions decreased from around 95% in 1970-2000 to 90% since 2005. We show and discuss this change in confidence in Appendix D. While Gender API’s average prediction confidence in our corpus is still high, this trend may pose a challenge for similar analysis in the future. Upon inspection of the data, we attribute this to the growing number of East Asian authors publishing in recent years. East Asian first names, when romanized, are more gender ambiguous. Gender API outperforms other gender lookup services, but still has lower overall confidence on names of East Asian origin (Santamaría and Mihaljević, 2018). In Mattauch et al. (2020), the authors explicitly exclude all authors with East Asian names from their name list during analysis, yet this accounts for the removal of more than 35% of their dataset. Rather than remove an entire group of authors from our data, we believe that representing each author name as a distribution of gender probabilities offsets some of the issues of increasing gender ambiguity in our corpus over time.

We also recognize the limitations of using author-paper pairs as our units of measure. We do not distinguish between a person who is a single author on a paper, and a person who co-authors with many others. This biases our data by over-weighting authors in papers with more authors. Similarly, in our analysis of collaboration, we take each combination of authors for a paper as a collaborating pair, which again over-weights papers with more authors. In the Computer Science corpus, we observe an increase in the average authors per paper over time, growing to approximately 3.0 authors per paper in the last two years. However, Computer Science papers are still generally authored by smaller groups of individuals in the lower single digits, and we believe the bias introduced by our usage of author-paper pairs or collaborating author pairs to be minimal.

Each author on a publication is also weighted equivalently in our analysis. We acknowledge that this discounts the special recognition extended to first authors, last authors, and single authors; we point readers to previous studies that have already demonstrated the distinctions between these groups (West et al., 2013).

Lastly, our projection of female author proportion uses data from the last 50 years to project more than 100 years into the future. We understand the inaccuracies of making such an extensive forecast with limited data. The goal of our projection is not to provide a definitive answer to the question of when gender parity will be reached among Computer Science authors; rather, the projection signals that even under optimistic growth, the gender gap will likely not close in the near future without some form of community or external intervention. Observed recent trends also suggest that the increase in female representation among Computer Science authors may be slowing in the last five years. The long range forecasts we show may not adequately capture changes on this shorter time scale. Our forecasts also do not reflect changes that would result from newly introduced or as yet unimplemented interventions.

5.2. Prior work

Inequality in gender representation is a well documented and studied issue in academia. Studies have shown that existent and perceived gender biases may affect many aspects of career and academic success, including but not limited to a woman’s choice of college major (Robnett, 2015), crediting in scientific publications (Feldon et al., 2017), access to mentorship (Decastro et al., 2014; Schluter, 2018; Moss-Racusin et al., 2012), rate of promotions (Clifton et al., 2019), opportunities for collaboration (Els, 2017), as well as publishing and citation trends (Mattauch et al., 2020; Mohammad, 2020). All of these factors can lead to imbalanced representation of women in certain fields of study.

With the increasing digitization of scholarly communication and availability of publication-related metadata, scholars have been able to better quantify inequality in authorship. Cohoon et al. (2011) analyzed 86,000 ACM conference papers and showed increasing representation of women authors publishing at Computer Science venues, which strongly correlated with increasing numbers of female Computer Science PhDs (Cohoon et al., 2011). West et al. (2013) analyzed 1.8 million papers from JSTOR, a large multi-disciplinary repository of academic literature, and revealed that although gender gaps are shrinking in academic publications, women were found to be significantly underrepresented as last and single authors. Elsevier, a large publisher of research articles, in an analysis of data from Scopus and ScienceDirect, reported the presence of gender imbalance among authors and inconsistent trends towards equal representation in different fields (Els, 2017). A study in 2018 confirmed continuing gender disparities among Nature Index journals, commonly considered some of the most reputable sources of academic literature, and in particular, limited representation of women among last authors, who are often perceived as more senior (Bendels et al., 2018). Our work demonstrates that the gender gap is persistent and relatively large among Computer Science authors, which is consistent with the results of these studies.

A study of gender bias in authorship conducted by Holman et al. (2018) projected the closing of the gender gap in various fields based on recent trends. Through analyzing 9.1 million articles from PubMed, the authors projected that gender parity would be reached in around 20 years in certain biomedical fields such as Molecular Biology, Medicine, or Biochemistry. Holman et al.’s analysis of a small corpus of Computer Science pre-prints from arXiv showed that gender parity in Computer Science will be reached in more than 100 years from the present (Holman et al., 2018). Also corroborating our estimate is related work from Way et al. (2016), which forecasts that gender parity in CS faculty hiring will be reached around 2075. Due to the long duration of faculty careers, parity in hiring would be expected to precede parity in publication and overall representation. Our results confirm and expand upon the results of this prior work. We use a significantly larger corpus of literature metadata to place the trends observed in Computer Science in the context of other fields of study. Additionally, we provide an assessment of co-authorship trends, which demonstrate a gap in cross-gender collaborations among CS authors.

Major strides have been made to reduce gender disparities. The presence of an overall structure of sexism in academia continues to be debated (Lundine et al., 2018; Boynton et al., 2018; Lundine et al., 2019), but many academic institutions recognize the issue and have sought to equalize admissions and hiring procedures. Evidence of movement towards more equitable representation in hiring and publication has been observed in some controlled settings (Williams and Ceci, 2015; Hengel, 2017; Ceci and Williams, 2011). How these observations translate into systemic change remain to be seen. Our results suggest, however, that the current pace of change in Computer Science will not result in a rapid closing of the gender gap.

6. Conclusions

We performed a large-scale analysis of the Computer Science literature (11.8M papers) to evaluate gender trends among authors. Based on trends over the last 50 years, the proportion of female authors in Computer Science is forecast to reach parity beyond the end of this century, and under different assumptions—it may take far longer. In this regard, Computer Science trails other fields of study, to which we may want to look for inspiration. We also observed lower than expected numbers of cross-gender collaborations, with a gap of approximately 50000 cross-gender collaborations per year in the last several years.

Unless a major shift occurs that changes the gender makeup of the Computer Science community, the authorship gender gap will likely persist for a long time. Given the pervasiveness of computing technologies in our daily lives, it is of utmost importance that the designers and builders of these technologies reflect the diversity of their users. Gender is one type of diversity among many that can be more easily assessed using the types of automated methods we employ. We hope that these findings will motivate members of the community to reflect upon the causes of these disparities, and provide evidence to back up policy decisions to change the status quo.

Acknowledgements.

We would like to thank Jonathan Borchardt, Matt Gardner, and Candace Ross for conducting the initial analysis that motivated this work. We would also like to thank Kyle Lo for methodological discussions and Ashish Sabharwal, Maarten Sap, Noah Smith, and Mark Yatskar for helpful comments on earlier drafts of this paper.

Appendix A Sensitivity analysis for parity projection

Figure 5 shows a sensitivity analysis over the equilibrium female author proportion parameter $\alpha$. This analysis shows the year in which parity is first reached at each equilibrium proportion; note that when $\alpha=0.5$, exact 50/50 parity is, by definition, never attained in finite time. We therefore report the time at which the female author proportion surpasses 0.45, within 10% of exact parity. When the equilibrium proportion is expected to favor women over men (above 0.5), the year in which parity is reached occurs earlier. Even with the aggressive projection that women will eventually author 90% of all publications, the expected year in which 50/50 parity will be reached is still around 2100.

Appendix B Average authors per paper

Figure 6 shows the average number of authors per paper over the years. Numbers of authors per paper have increased significantly in recent years, especially in fields dominated by large-scale experiments, like Physics, Biology, and Medicine. Although Computer Science has also seen an increase in contributing authors per paper, this growth is much slower relative to other scientific fields. This stands in contrast to fields that are closer to the humanities, such as Art, History, or Philosophy.

Appendix C Assumption of logistic fit

We assume the growth of the proportion of female authors observes logistic behavior. The choice of logistic fit is natural because the proportion of female authors is necessarily bound by 0 and 1, which is a feature of logistic growth curves. In this Appendix, we further justify the choice of logistic fit by comparing it against linear and exponential fits, which could be better descriptors of rates of change in the short term. For each field of study, we fit a linear, exponential, and logistic curve to the proportion of female authors over time, beginning in 1970, when the trends primarily show consistent growth. We compute the RMSE of each fit and provide this in Table 2. For all fields of study, the RMSE of logistic fit is the lowest of the three curve types.

Appendix D Uncertainty in author gender

The average confidence of Gender API results averaged over the authors in each year is shown in Figure 7. The average confidence of Gender API is decreasing over time in our corpus, and experienced a drop after 2000 to around 90% in recent years. We speculate about the cause of this drop in §5.1. Analyses such as ours may become increasingly difficult to perform in the future unless the datasets behind gendering services are improved.