The Chaperone Effect in Scientific Publishing

TL;DR

This paper quantifies the 'Chaperone Effect' in scientific publishing, showing how experience within a journal influences author seniority and impact, especially in high-impact venues and certain scientific fields.

Contribution

It introduces a quantitative measure of the 'Chaperone Effect' and analyzes its variation across disciplines and its impact on publication success.

Findings

The chaperone effect is stronger in medical and biological sciences.

Experienced authors tend to publish in higher-impact journals.

The effect influences the career trajectory of scientists.

Abstract

Experience plays a critical role in crafting high impact scientific work. This is particularly evident in top multidisciplinary journals, where a scientist is unlikely to appear as senior author if they have not previously published within the same journal. Here, we develop a quantitative understanding of author order by quantifying this 'Chaperone Effect', capturing how scientists transition into senior status within a particular publication venue. We illustrate that the chaperone effect has different magnitude for journals in different branches of science, being more pronounced in medical and biological sciences and weaker in natural sciences. Finally, we show that in the case of high-impact venues, the chaperone effect has significant implications, specifically resulting in a higher average impact relative to papers authored by new PIs. Our findings shed light on the role played by experience in publishing within specific scientific journals, on the paths towards acquiring the necessary experience and expertise, and on the skills required to publish in prestigious venues.

Full text

\verticaladjustment

-2pt

\dropcap

Science as an institution is highly stratified (1), and anecdotal evidence that scientific high-achievers are often protégés of accomplished mentors, supports the notion that scientific status is passed along through lineages of prominent scientists (2, 3, 4). While single-topic studies like the mathematical genealogy project document such bonds between renowned scientists (5), there is less quantitative understanding of the role of apprenticeship in scientific publishing and of how scientific excellence is passed along between generations (6, 5). Here we quantify a key aspect of this ‘chaperone effect’ by considering how inexperienced scientists transition into senior status given multiple publications within the same scientific journal. We illustrate that the chaperone effect has different magnitude for journals in different branches of science, the effect being more pronounced within medical and biological sciences and weaker for the natural sciences. For high-impact multidisciplinary journals, a scientist is unlikely to appear as senior author if he or she has not previously published within the same journal. Our findings shed light on the role played by scientific training to acquire the necessary experience, expertise and skills to publish in venues characterized by a strong chaperone effect.

In general, there are a wealth of indications that young scientists who interact with successful mentors have a higher probability of achieving success later in their careers. For example, an improbably large fraction of Nobel laureates were trained by other laureates (1, 7). Beyond the core skill of learning to select relevant scientific questions and providing meaningful answers, an important aspect of career success rests on publishing in prestigious venues. Here we focus, not on mentorship directly, but on an important facet of the mentorship process: Experience with publishing within a specific journal.

The order of authors on multi-author scientific articles provides important signals regarding the role of each scientist in a project (8, 9). For example, in biological and increasingly in physical sciences typically, the first author is an early-career scientist who carries out the research, while the last author is a mentor-figure who plays a role in shaping the research, establishing the paper’s structure, and corresponding with journal editors (10, 11). Middle authors generally play more specialized roles, such as contributing statistical analyses. This division of labor is often symbiotic; it has recently been shown that junior researchers tend to work on more innovative topics but need mentorship (12, 13). Further, high impact works are often performed by multiple authors whose composition is usually heterogeneous in terms of experience (14, 15, 16, 17). In this work, we use author-order to study the role of experience in crafting scientific work (6) by analyzing the dynamics of scientific multi-author publications (10, 11). Such sequences provide a ‘petri dish’, unveiling the patterns that increase the rate of acceptance for some authors. To unravel how the dynamics of these sequences vary across the sciences, we explore the extent to which the principal investigator (PI) of a paper has previously published in the same journal as a junior author. Thus, we address a question which is often asked by scientists: ‘Can you publish in Nature if you have never published in Nature before!?’. Note that here we take Nature as an example of a journal with high impact factor. However, our analysis spans multiple journals, as described in Materials and Methods.

We consider $6.1$ million papers published between 1960 and 2012 in $386$ scientific journals; covering the fields of mathematics, physics, chemistry, biology, and medicine (see Materials and Methods for data processing and name disambiguation). Included are the top $3$ multidisciplinary journals: Nature, Science, and PNAS. In our analysis, we assume that the principal investigator (PI) is listed last in a paper’s author list, a common practice in many scientific fields (18, 10, 11). Note, however, that our analysis is not affected if the author list of some papers does not mirror seniority roles (see Materials and Methods). For all papers in each journal, we divide PIs into 3 categories: new PIs are those who have not published previously in that specific journal, chaperoned PIs are those who have appeared before only as junior (non-last) authors, and established PIs are those already previously listed as a last author in the journal (see Fig. 1a). By definition, the last author on any given publication can be classified only in one of these categories. For example, F.J. Weissing’s first paper in Nature was as last author, so he is labeled as a new PI in Nature for that year (1999). In 2007, Weissing published in Nature as last-author again, but because of the previous publication, we categorize him as as an established PI in Nature in 2007. This 2007 Nature paper was co-authored by three other scientists, one of them being O. Leimar. A year later, Leimar published a paper as last author in Nature, and is therefore marked as a chaperoned PI in Nature for that year. In Figure 1b we show the fraction of new, established, and chaperoned authors over time for three scientific journals.

The proportion of these three kinds of author is substantially different depending on the journal. New England Journal of Medicine (NEJM) is an example of a journal where the highest fraction of senior authors is new (Fig 1b, red line), signaling that repeat authorship is less common, i.e. the medical community tends to submit only their most groundbreaking work to this high-impact general interest journal. As a point of contrast, we show Physical Review D as an example of a journal where established PIs are predominant, a tendency which increases over in time (Fig 1b, blue line). This picture arises when some authors specialize in writing for a particular disciplinary journal, leading to a large fraction of repeated names in the PI spot. In the bottom panel we show Nature, a journal with an interdisciplinary audience, which has undergone a strong change over the past 10 years, with the fraction of new authors dropping significantly. This indicates that it is becoming increasingly rare to publish as the senior author in Nature without previous publishing experience in the journal. A possible explanation for this development is an increasing number of authors specializing in writing papers for high-impact general audience journals, eschewing the more traditional pattern of publishing primarily in specialized journals and sending only selected results to high-impact multidisciplinary journals.

To understand the role of journal-specific experience, we investigate the chaperoned authors more closely. Chaperoned authors are senior authors, who have published in the journal previously as non-last authors (see Fig 1a). Due to prior experience with the process, chaperoned PIs already have gone through the intensive process of preparing a manuscript for a high-impact journal and absorbed tacit knowledge on how to frame the message appropriately for the journal audience, strike the right tone in the cover letter, structure the supporting information, the subtleties of how to constructively interact with editors, mastering layers of information that is usually invisible to those reading a paper. Hence, the senior author acts as a chaperone simply through guiding the submission process. Having experienced the entire publication process once increases the chances of publishing in similar journals again, since the author is familiar with their particular idiosyncrasies. In Fig 1b, the chaperoned fraction hovers at around $0.1-0.2$ for all three journals, but showing an increasing trend over time for NEJM and Nature. Thus in both NEJM and Nature, high impact journals with a wide audience, the fraction of new authors decreases over time, while the fraction of chaperoned authors slightly increases. In other words, it is becoming harder to publish in Nature without having published in Nature before.

We quantify the chaperone effect $c$ for a journal by comparing the number of authors that over time have made the transition from a non-last position to the last position in the author order, with the number of last-authors that within the journal have never made such a transition over time. In other words, we compare the proportion between new and chaperoned PIs. The chaperone effect captures the difficulty associated with publishing in a journal without previous experience with that journal. A chaperone effect of $c=1$ implies that there is a balance between new and chaperoned authors. If the chaperone effect for a journal is greater than one ($c>1$), it means that the fraction of new authors is smaller than the fraction of chaperoned authors and that, in order to publish in the journal in question, it is important to have a senior author act as a chaperone. Conversely, if a journal has a chaperone effect smaller than one ($c<1$), publication is easier for new authors. The specific value of $c$, however, is affected by field specific characteristics and publishing conventions, like typical team size and individual productivity (19). It also does not take into account the fact the an author can make the transition from non-last to last position randomly. For these reasons, $c$ of different fields cannot be directly compared.

To correct for these caveats in the quantification of the chaperone phenomenon through $c$ and to be able compare the importance of apprenticeship across the sciences, we compare the observed values of new and chaperoned, with those occurring in two null models (20, 21). Firstly, we consider a system where the ordering of author names is not relevant (22). Therefore, we compare $c$ to $c_{\text{random}}$, where $c_{\text{random}}$ is the ratio obtained in a null model where we have randomly permuted the order of author names in each paper. We call $C=c/c_{\text{random}}$ the magnitude of the chaperone effect. Note that the magnitude $C$ cannot be affected by team-size and individual volume productivity, as these are preserved in the randomization. However, $C$ does capture significant changes in the order of authors in respect to the random ordering. In general, the chaperone phenomenon occurs when $C>1$, i.e. when the transition non-last$\rightarrow$last is more frequent than the appearance of new authors in the last position in a statistically significant way. Secondly, in some fields (e.g. mathematics), alphabetical author-sorting is an important convention. Therefore, we also compare $c$ to $c_{\text{alphabet}}$, which is based on a system where all author-lists are sorted alphabetically (23). Based on this second model, we construct $C_{\text{alphabet}}=c/c_{\text{alphabet}}$. Values of $C_{\text{alphabet}}$ are typically smaller than one (see SI Appendix section 1.). In a nutshell, the deviation of $c$ from $c_{\text{random}}$ and $c_{\text{alphabet}}$ provides the magnitude of the chaperone effect, stripped of any confounding effects (see Methods).

In Fig. 2, we show the distribution of $C$ as well as $C_{\text{alphabet}}$ for the five fields mentioned above and for interdisciplinary journals. This figure is in line with the collective intuition about ‘the purity of sciences’ (24). Mathematics shows very few signs that experience influences the transition between junior and senior levels. This is likely in part due to the fact that authorship conventions in mathematics dictate alphabetical order for all publications (23). We see the magnitude of the chaperone effect growing across physics, chemistry, and medicine, with the strongest effect within biology and general-topic journals. For these fields, there is a clear relationship between having published in them as a junior researcher and the probability of publishing in them as PI, illustrating that experience with publication is important for transitioning between junior and senior authorship within high-impact journals.

Assessing the existence of an unbalanced proportion of chaperoned and new last-authors prompts an important question: How does publishing in a journal as a non-last author impact your odds for one day publishing as last author? Since we do not have access to statistics for rejected papers, we are unable to answer that question exactly. We can, however, answer a closely related question, namely: How does the probability of transitioning to last author change as a function of number of occurrences as a non-last author? In Fig. 3a we see that, in the case of Nature, this probability grows significantly from $10\%$ after one publication to nearly $20\%$ after four publications as non-last author (for the study of chaperone effect in PNAS, see SI Appendix figs. S3 and S4). In contrast, the same transition probability is $25\%$ in the case of the highly disciplinary Physical Review D, and does not change with additional publications as non-last author.

There is a critical aspect of the chaperone effect that we have not yet explored: does experience with publication within a certain journal play a role in the scientific impact of subsequent papers as PI published in the same venue? Could it be that new, chaperoned and established last-author papers receive different levels recognition from the scientific community? Our initial hypothesis was that papers authored by new PIs might have higher impact, since their lower odds of being published might signal a higher significance of the reported discoveries for the scientific community. To test this hypothesis, we quantified the impact of each paper by measuring $c_{5}$, its citations after 5 years from publication, a measure that is not affected by the specific field citation dynamics (25). This allowed us to directly calculate the average impact over time for three categories of papers, those with chaperoned, established and new PIs. In the case of Nature the result is striking (Fig. 3b). We find that papers with established and chaperoned PIs have indistinguishable impact. Contrary to what we expected, however, papers authored by new PIs in Nature receive on average only half the citations of papers authored by chaperoned and established PIs, indicating a systematically lower scientific impact. The same pattern is observed in the entire group of interdisciplinary journals, suggesting this pattern is consistent in these venues with high selection pressure and only a small fraction of all scientists manage to publish as PI. In more specialized field-specific journals, a difference can be also present, but the differences between the three categories of authors tend to be smaller (see SI Appendix). Thus, our findings suggests that experience of publishing within specific journals can play an important role in acquiring long-term scientific impact.

Taken together, our results add new piece to the puzzle of how mentor-protegee relations function more generally (6, 26, 27, 28, 29, 5), where a full picture of the relation will also draw on understanding of how teams are assembled and produce knowledge (30, 14). In this sense, additional research is needed to understand the complex processes that drive the differences between new and chaperoned authors. By focusing on the role of experience within journals and fields, we deliberately average over authors with very different levels of success and do not account for the fact that good proteges tend to find good mentors; nor do we include the fact many young authors leave science altogether. Therefore, it is important to stress that our results are not designed to answer the deeper questions on mentor-protege rules, but point to the general structures in how knowledge needed to write for certain journals is different across the sciences, with high-impact, interdisciplinary journals showing a particularly strong effect.

Thus, while the available data here do not allow us to strictly pinpoint which facet of experience is most important in order to succeed in science, or which share of a senior authors’s apprentices are successfully chaperoned (31), we have demonstrated that the chaperone effect does indeed exist, showing that the ability to publish in certain venues is something that junior scientists learn from senior colleagues. Further, we have demonstrated that apprenticeship is not just about membership in the exclusive club of having-published-in-Nature or another prestigious journal, but papers by chaperoned authors have greater scientific impact than papers by new PIs. In addition, we show that the magnitude of the chaperone effect varies across scientific fields (24). The chaperone effect is most strongly expressed in prestigious multidisciplinary journals, demonstrating that the highly specialized skill-set required in order to publish in these venues is passed along more strongly than any field-specific expertise.

\matmethods

Data

We use publication data provided by the Web of Science database (www.webofknowledge.com), purchased for research purposes by some of the authors of this publication in 2013. The database includes several types of scientific outputs such as articles, letters, reviews, editorials and abstracts from 1898 to 2012 across more than 22,000 scientific journals from broad domains, resulting in a set of more than 50 millions papers. For each paper, the dataset includes more information on the date of publication (month, day, year), the journal name and journal issue, author names with the order they appear on the article, their affiliations, and the references towards past articles indexed in the database. For Nature we have downloaded the full publication history using the Nature opensearch API.

For our analysis, we focused on publications from 1960 to 2012 published in interdisciplinary journals (Nature, Science, and PNAS), as well as in journals associated to five distinct scientific fields: Medicine, Biology, Mathematics, Chemistry and Physics. To identify the journals belonging to each category, we first parsed dedicated Wikipedia pages containing lists of journal names associated to specific scientific fields and then matched these with the journals in the database (32). In total we identified 97 biology, 337 medicine, 243 physics, 248 mathematics, 138 chemistry, and 3 interdisciplinary journals.

Next, we extracted the publications associated to each of these categorized journals. To ensure to deal with original research, we collected only publications labeled as Articles, Letters and Reviews and that did not have a title containing the terms comment, reply, errata, or retracted article. Moreover, in order to have enough statistics, only the categorized journals fulfilling the following criteria were taken into account for our analysis:

• •

The collected publications associated to the journal are spanning a period of at least 10 years;

• •

At least 1,000 collected publications were published in the journal overall;

• •

At least 100 collected publications were published each year in the journal.

After this preprocessing, our data amounts to (i) 795,558 publications from 40 journals in biology, (ii) 1,350,936 publications from 128 journals in medicine, (iii) 1,753,641 publications from 117 journals in physics, (iv) 208,223 publications from 26 journals in mathematics, (v) 1,341,150 publications from 72 journals in chemistry and (vi) 251,294 publications from Nature, Science and PNAS. Data about the proportion of new, established and chaperoned PIs over time, and the values of $c$, $C$ and $C_{alphabet}$ are provided for each journal on GitHub (https://github.com/SocialComplexityLab/chaperone-open). Raw data from Web of Science cannot be shared publicly on the web, but we offer the possibility to reproduce our results starting from raw records by spending a research visit at Northeastern University or Central European University where the data is accessible. Data about the journal Nature can be downloaded for free from Nature opensearch (https://www.nature.com/opensearch/).

Author name disambiguation

We formatted all author names present on the collected publications to lower case and converted their names into their first letter only. An author named "John Smith" or "Mary Suzy Johnson" would thus be converted to the format "smith,j" or "johnson,ms", respectively. We considered the sequence of publications within the same journal and authored by an identical formatted name to correspond to a same individual. We expect errors induced by homonyms, i.e. distinct individuals that share the same formatted name, to be low as we only compare names within the same journal. An error can thus only occur if two distinct individuals share the same formatted name and evolve in the same scientific field, i.e. the same journal, which is already an accurate disambiguating feature (33).

Robustness of results to alphabetic ordering

In certain scientific fields it is common to order authors alphabetically (18). As such, to understand how this affects the results, we perform two versions of our analysis: one, taking all publications into account, and two a version where we have disregarded publications where authors are alphabetically ordered. This removes $17.7\%$ of all publications within Biology, $14.4\%$ within Medicine, $30.9\%$ within Physics, $75.1\%$ within Mathematics, $23.3\%$ within Chemistry, and $20.8\%$ within interdisciplinary journals. Note that these numbers include publications where the authors are ordered by choice, but also publications where this occurred by chance. Nonetheless, our conclusions are robust for both data sets, consistently with the result shown in Fig. 2 that there is a significant difference between observed $C$ and that of the alphabetical null model $C_{alphabet}$ (see SI Appendix fig. S5).

\showmatmethods

\acknow

This work was supported by Air Force Office of Scientific Research grants FA9550-15-1-0077 and FA9550-15-1-0364 (A.-L.B. and R.S.), The European Commission, H2020 Framework program, Grant 641191 CIMPLEX, The Templeton Foundation (R.S., A.-L.B.), and the ITI project ‘Just Data’ funded by Central European University (R.S.), The Villum Foundation (S.L.), The Independent Research Fund Denmark (S.L.). R.S. thanks Michael Szell for useful discussions and feedback, and Alex Gates for support with the Web of Science data.

\showacknow