Be taught Article

Leer ORCID ProfileMary C. Murphy, Leer ORCID ProfileAmanda F. Mejia, Leer ORCID ProfileJorge Mejia, Leer ORCID ProfileXiaoran Yan, Leer ORCID ProfileSapna Cheryan, Leer ORCID ProfileNilanjana Dasgupta, Leer ORCID ProfileMesmin Destin, Leer ORCID ProfileStephanie A. Fryberg, Leer ORCID ProfileJulie A. Garcia, Elizabeth L. Haines, Leer ORCID ProfileJudith M. Harackiewicz, Leer ORCID ProfileAlison Ledgerwood, Leer ORCID ProfileCorinne A. Moss-Racusin, Leer ORCID ProfileLora E. Park, Leer ORCID ProfileSylvia P. Perry, Leer ORCID ProfileKate A. Ratliff, Leer ORCID ProfileAneeta Rattan, Leer ORCID ProfileDiana T. Sanchez, Leer ORCID ProfileKrishna Savani, Leer ORCID ProfileDenise Sekaquaptewa, Leer ORCID ProfileJessi L. Smith, Leer ORCID ProfileValerie Jones Taylor, Leer ORCID ProfileDustin B. Thoman, Daryl A. Wout, Leer ORCID ProfilePatricia L. Mabry, Leer ORCID ProfileSusanne Ressl, Leer ORCID ProfileAmanda B. Diekman, and Leer ORCID ProfileFranco Pestilli

  1. Edited by Susan T. Fiske, Princeton University, Princeton, NJ, and accepted July 27, 2020 (purchased for review December 7, 2019)

Significance

Science is rapidly changing with the present movement to beef up science centered largely on reproducibility/replicability and start science practices. Thru network modeling and semantic evaluation, this article offers an initial exploration of the structure, cultural frames of collaboration and prosociality, and representation of girls folk in the start science and reproducibility literatures. Community analyses indicate that the start science and reproducibility literatures are rising slightly independently with few same old papers or authors. Open science has a more collaborative structure and entails more specific language reflecting communality and prosociality than does reproducibility. At last, ladies folk publish more steadily in excessive-location author positions within start science in contrast with reproducibility. Implications for cultivating a various, collaborative tradition of science are mentioned.

Abstract

Science is present process rapid switch with the movement to beef up science centered largely on reproducibility/replicability and start science practices. This moment of switch—in which science turns inward to seek its methods and practices—offers an opportunity to address its ancient lack of diversity and noninclusive tradition. Thru network modeling and semantic evaluation, we provide an initial exploration of the structure, cultural frames, and girls folk’s participation in the start science and reproducibility literatures (n = 2,926 articles and convention proceedings). Community analyses counsel that the start science and reproducibility literatures are rising slightly independently of every assorted, sharing few same old papers or authors. We next seek whether or no longer the literatures differentially incorporate collaborative, prosocial beliefs that are identified to rob members of underrepresented groups more than self reliant, winner-takes-all approaches. We uncover that start science has a more linked, collaborative structure than does reproducibility. Semantic analyses of paper abstracts indicate that these literatures have adopted assorted cultural frames: start science entails more explicitly communal and prosocial language than does reproducibility. At last, per literature suggesting the variety advantages of communal and prosocial capabilities, we uncover that ladies folk publish more steadily in excessive-location author positions (first or last) within start science (vs. reproducibility). Furthermore, this finding is extra patterned by team size and time. Females are more represented in greater groups within reproducibility, and girls folk’s participation is rising in start science over time and lowering in reproducibility. We enact with actionable ideas for cultivating a more prosocial and various tradition of science.

At the present moment, science is present process a “revolution” to better itself (1). The purpose of this revolution is brave. At its core, the movement to beef up science encompasses two major objectives: 1) belief the concerns, weaknesses, and reproducibility of previous scientific processes and findings (e.g., evaluating the strength of the proof) and a pair of) bettering learn practices by scheme of greater rigor and transparency (e.g., start sharing of recordsdata, code, resources; standardized statistical procedures; preregistration). As with every revolution, a time of unrest can additionally be a time of opportunity. Certainly, researchers inquisitive referring to the efforts to beef up science have acknowledged a gender diversity affirm (2, 3), and this time of reform provides the chance to reinvent scientific tradition in a more inclusive mode. If the movement to beef up science perpetuates the aged scientific tradition that prioritizes self reliant, dominant, or adversarial values, it risks continuing to scurry away many gifted participants at the margins, feeling unwelcome and excluded (4)—exacerbating a global affirm that the sciences try to clear up (58). In its efforts to beef up its methods and replicability, we questioned whether or no longer science would possibly almost certainly additionally be reaching improvements in the gender representation and inclusivity of the movement itself. This text applies cultural and network evaluation to seek the rising cultures in the movement to beef up science—particularly in the reproducibility and start science literatures—and to investigate the representation of girls folk in these rising subcultures. We discuss implications of these assorted cultural avenues for science going forward.

In cultural analyses, the actions and cognitions of individuals each upward push from and produce the norms and practices of groups and institutions (9). Extra, the “who” and the “how” of cultural practices are inextricably intertwined: “how” a subculture operates influences “who” engages in the subculture, and “who” engages in the subculture influences “how” a subculture operates. The cultural practices of the present scientific reform movements have an effect on who engages. The rising reform movements have their roots in the broader tradition of science, abilities, engineering, and math (STEM) that can aid as a barrier to the inclusion and vogue of girls folk (1012). The tradition of science has long valued particular particular person brilliance, competition, and a winner-take-all mannequin of success (13). In specific, individuals in and out of doors of STEM glimpse STEM fields as affording more alternatives for particular particular person success and fulfillment than for prosociality and collaboration (14).

The scientific be conscious of rewarding particular particular person fulfillment has almost certainly unwittingly fostered a more self reliant, aggressive tradition that ignores and almost certainly even disincentivizes cooperation (15, 16). These cultural practices have implications for who joins and advances within scientific fields. For instance, the perceived lack of prosocial and collaborative tradition in STEM has been shown to deter ladies folk particularly (14, 17). Certainly, the presence of collaborative practices and prosocial capabilities is seemingly to be particularly necessary in fields centered on scientific reform: critiquing established authors or practices—no subject how effectively intended or delicately acknowledged—is customarily interpreted as criticism and puts the critiqued in a defensive situation.

The function of critic is seemingly to be particularly dangerous and unappealing to feminine scientists. First, ladies folk would possibly almost certainly feel much less ready to converse dissent (particularly when in the numerical minority) against established figures, because this conflict-inclined stance violates gender function expectations (18). Females who’re perceived as self-promoting or aggressive face more adverse evaluations than their male counterparts (19); thus, enticing in evaluations or debates can elicit more backlash towards ladies folk than men, and the mere anticipation of backlash can inhibit ladies folk’s engagement in these spheres. Second, ladies folk would possibly almost certainly use a collective scheme for pragmatic and principled causes. Pragmatically, there would possibly be psychological security in numbers (2022), and girls folk’s evaluations is seemingly to be seemingly to be provided and listened to when they’re portion of a greater scientific team. Extra, because combative and adversarial behaviors are perceived as masculine, ladies folk is seemingly to be much less socialized to rob in these behaviors than men and/or see them as off-inserting and never more seemingly to be productive (23). In notion, a collectivist orientation would possibly almost certainly detest challenges to the establishment when framed as for the coolest thing referring to the challenger (i.e., gaining recognition) moderately than for the collective ethical (i.e., bettering and advancing science).

On the change hand, we scheme attention to every other causal pathway as effectively: subcultures that consist of a greater percentage of girls folk (or assorted underrepresented neighborhood members) would possibly almost certainly rob in assorted practices than more homogenous subcultures. For instance, legislative bodies that consist of greater proportions of girls folk legislators rob more with insurance policies linked to education and health care (2426). Culture is a cyclical task, and thus greater inclusion and vogue of girls folk foster norms and behaviors that in flip can make contributions to rising gender diversity (6, 27, 28).

The movement to beef up science, prior to now, is seemingly to be characterized by two contrasting motifs—each aimed to beef up science. One level of curiosity centers on the review of the reproducibility and replicability of previously revealed scientific outcomes. We show that the Nationwide Academies of Sciences, Engineering, and Tablets has easiest no longer too long ago formalized a distinction between reproducibility and replicability (29). Sooner than this formalization, the 2 phrases had historically been extinct with assorted conventions in assorted fields, with a occurrence of the term reproducibility (2936). For this purpose, our evaluation (that makes use of historical recordsdata across fields) does no longer separate the 2; as a change, at some level of the narrative, we use the term “reproducibility” to discuss to the literature that we analyze.* A 2nd scheme aimed to beef up science includes “start science” practices that facilitate the sharing and reuse of learn property (e.g., recordsdata, code) in uncover to beef up rigor and flee up the charge of scientific discovery (3739). For shorthand, we discuss to those two literatures as “reproducibility” and “start science.” Certainly, each literatures purpose to beef up science, are led by scientists, and rob in deep evaluation and critique of present scientific practices whereas providing steerage and ideas for be taught the components to beef up scientific practices. Here, we explored whether or no longer the reproducibility and start science literatures articulate assorted 1) collaborative structures, 2) explicitly prosocial foci, and 3) engagement of feminine scientists. We anticipated that this initial investigation would indicate proof of assorted rising cultures in the reproducibility vs. start science literatures—with implications for the future representation and practices of these movements.

Our team performed network analyses of the start science and reproducibility literatures and chanced on that these literatures have few same old papers and authors—suggesting these enchancment approaches have developed slightly independently from each assorted. Given this, we in contrast these literatures for hallmarks of collaborative and prosocial tradition. We uncover a more interconnected authorship network within start science in contrast with reproducibility, and semantic text analyses of article abstracts indicate that the start science and reproducibility literatures appear like adopting assorted specific cultural frames. Open science entails considerably more language that reflects the cultural values of prosociality in contrast with reproducibility. We then seek the configuration of girls folk’s participation in these literatures. We uncover patterns of girls folk’s participation per the theoretical thought that ladies folk’s participation is much less constrained in additional collaborative and prosocial cultures (i.e., in start science than in reproducibility). Females scholars are seemingly to buy excessive-location author positions (taking the first or last author situation) within start science in contrast with reproducibility (see Fig. 3); extra, ladies folk’s excessive-location authorship occurs much less steadily in smaller groups within reproducibility (in contrast with start science). In greater groups—that can almost certainly offer greater collective security or communal purpose—there would possibly be diminutive distinction in ladies folk’s representation in management roles between the 2 literatures. At last, we uncover that ladies folk’s participation in excessive-location authorship positions is rising over time in start science, whereas it is lowering in reproducibility.

Taken together, we uncover that despite present controversies (2), the start science level of curiosity of the movement to beef up science has the seed of an interconnected and prosocial tradition that, if extra cultivated, would possibly almost certainly continue to entice greater participation by ladies folk. We imagine that the collaborative, forward-having a see level of curiosity of start science has the capability to facilitate greater diversity and inclusiveness. Whereas our level of curiosity on author gender listed right here used to be motivated, in portion, by the ability to be conscious validated, automated coding methods (that are extremely reproducible) to resolve author gender, we would on the opposite hand predict identical findings for scholars from assorted underrepresented groups. When fields are more adversarial and never more prosocial, participants from underrepresented groups (in conjunction with ladies folk) is seemingly to be much less motivated to rob (40) due no decrease than in portion to the vitality dynamics described above. In disagreement, fields that emphasize collaborative and prosocial norms encourage greater participation amongst underrepresented groups (41). It wishes to be famed that every adversarial and collaborative cultures can rob in rigorous debate and criticism. On the change hand, collaborative cultures can give you the money for more positive criticism, which is an indicator of ethical, forward-thinking science and what all scientists request of friends in the field. If we use to beef up and advance the field of science, then the onus is on investigators to nurture a convention that attracts and retains a diversity of oldsters (4244).

Outcomes

We performed each network science and semantic text analyses to construct the structural landscape and cultural foci of the start science and reproducibility literatures and girls folk’s participation in them. To enact so, our team analyzed recordsdata from Microsoft Academic Graph (MAG) (45), consisting of two,926 scientific articles and convention proceedings (hereafter customarily known as “papers”) revealed between 2010 and 2017 that included “start science” or “reproducibility” as a field of seek code (Concepts and SI Appendix). This sample consisted of 879 start science papers and a pair of,047 reproducibility papers. Finest 2.3% of papers shared “start science” and “reproducibility” field codes, suggesting these approaches are increasing slightly independently (see SI Appendix for more particulars).

Open Science and Reproducibility Fluctuate in Their Community Neighborhood Structures.

We analyzed a total of three,157 recurring article author identification numbers (IDs) in the start science literature and 8,766 in the reproducibility literature. We built two collaboration networks the use of these author IDs from MAG (Fig. 1). Nodes in these networks symbolize scientific articles; edges symbolize shared authorship such that two nodes fragment an edge if no decrease than one author appears to be like in each papers (see Concepts for particulars). Outcomes revealed that the start science network contained 879 nodes and 389 edges, whereas the reproducibility network contained 2,047 nodes and 856 edges. Importantly, the start science network is more edge-dense (0.101%) than the reproducibility network (0.041%)—demonstrating a greater stage of interconnectedness, which suggests a more dense collaborative network at some level of the start science literature (one-sided Fisher’s proper test: P < 0.001).

Fig. 1.

Differences in author neighborhood structure: start science (A) vs. reproducibility (B). Every circle, or node, represents a scientific article. Articles fragment an edge (line connecting two nodes) if no decrease than one author appears to be like in each papers. Whereas networks in each literatures are slightly sparse, the start science literature has fashioned a greater collaboration network (i.e., this neighborhood structure is seemingly to be considered by the neighborhood of extremely linked nodes in the center of the visualization), in contrast with the reproducibility network. Files had been visualized the use of Gephi (46).

We additionally performed a linked parts evaluation of every literature (47, 48) to measure the stage of isolation of particular particular person subnetworks of papers within each literature (Concepts). Outcomes show that the reproducibility network (1,641; 0.80 parts per article) contains more isolated articles (sharing fewer authors) than the start science network (661; 0.75 parts per article). This parts evaluation signifies that the reproducibility literature’s network is more fragmented. Inspecting the factor size differences of the 2 networks as every other indicator of connectedness, we uncover that the everyday factor size (ACS) is additionally greater for the start science network (ACS: 1.33 vs. 1.25). Fig. 1 visualizes the 2 networks to facilitate interpretation of the noticed network connectedness and fragmentation differences between the 2 literatures. In sum, the start science literature used to be chanced on to have a greater selection of connections (shared authors) between papers and the reproducibility literature contains more isolated and smaller paper networks—and these differences between the 2 literatures are statistically necessary (P < 0.01, as reported above). As a robustness check, we performed the equivalent analyses as an alternative of all solo-authored papers. Outcomes revealed that these findings are well-known to this change evaluation (see SI Appendix for particulars).

Semantic Textual disclose material Analyses Counsel That the Explicit Cultures of the Open Science and Reproducibility Literatures Are Diverse.

The use of a validated text-mining dictionary (49), we measured the presence of communal and prosocial constructs (e.g., make contributions, aid, support, nurture; see SI Appendix, Desk S2 for the list of constructs extinct) in the abstracts of the papers from each literatures. We excluded papers with out a in the market summary and participants with non-English titles. The resulting dataset included 595 start science papers and 1,169 reproducibility papers. Within the start science dataset, 76% of the articles extinct words linked to communal and prosocial constructs, whereas in the reproducibility dataset, easiest 44% of the articles did (two-sided test for equality of binomial proportions, P < 0.001). We computed the “prosocial word density” (PWD) within each dataset because the percentage of words in each summary that reflect communal and prosocial constructs (Fig. 2 and Concepts). The starting up science abstracts included more communal and prosocial words than the reproducibility abstracts (start science: mean PWD of two.4%, median PWD of 1.8%; reproducibility: mean PWD of 0.9%, median PWD of 0.0%). A two-sided permutation test for differences in the mean and median PWD in each dataset presentations that the start science literature entails considerably more frequent use of communal and prosocial words than does the reproducibility literature (P < 0.001 for mean and median PWD). Thus, we uncover that abstracts in the start science literature consist of considerably more words linked to communality and prosociality than those in the reproducibility literature.

Fig. 2.

Distribution of communal and prosocial word density of abstracts in the start science and reproducibility literatures. Abstracts in the start science literature consist of considerably more words linked to communality and prosociality than those in the reproducibility literature.

An change hypothesis is that these textual differences are merely driven by disciplinary field. To seek this possibility, we stratified the mannequin by academic field of seek (i.e., computer science, engineering, medication) and chanced on identical effects (see SI Appendix, Fig. S5 for particulars). Thus, the finding that start science incorporates more explicitly prosocial language in contrast with reproducibility is well-known to disciplinary field.

Females’s Participation Is Otherwise Patterned in Open Science and Reproducibility.

Females are seemingly to be represented in excessive-location author positions in start science (vs. reproducibility).

Females scholars are considerably seemingly to be represented in excessive-location author positions (i.e., first or last author situation) in the start science literature than in the reproducibility literature. Fig. 3 shows gender representation for the start science and reproducibility literatures for single- and multiauthored papers. The most effective-authored subset entails 255 start science papers and 342 reproducibility papers, whereas the multiauthored subset entails 624 start science papers and 1,705 reproducibility papers. As a result of assorted field conventions, we have in thoughts a pupil to take care of a excessive-location authorship situation if they buy either the first or last author situation within a multiauthored paper.

Fig. 3.

Gender representation in excessive-location author positions (first or last) in start science and reproducibility. (A) Single-author papers by gender. Females are underrepresented in single-authored papers in each the start science and reproducibility literatures, relative to gender parity. (B) Excessive-location positions in multiauthor papers by gender. Females are underrepresented in excessive-location author positions in each literatures (relative to gender parity) however have greater representation in start science (with 47% with identified feminine first or last author and 12% with identified feminine first and last author) in contrast with the reproducibility literature (with easiest 34% with identified feminine first or last author and easiest 5% with identified feminine first and last author).

We first analyzed single-author papers with identifiable author gender (we extinct an algorithm that employs census recordsdata to classify author names into the gender binary [SI Appendix], whereas acknowledging that gender is a complicated and multidimensional social believe). As in scientific publishing more broadly (5052), outcomes revealed that, total, ladies folk are considerably much less seemingly than men to publish single-author papers in each literatures. An proper one-sided Binomial test indicated that the percentage of feminine single authors is 33.0% in the start science literature and 28.1% in reproducibility; each are decrease than 50%—the percentage that can almost certainly stamp gender parity (P < 0.001 for every assessments). This implies that ladies folk are equally engaged with each topic build in single-author roles, even supposing underrepresented in each literatures in contrast with their single-author male colleagues.

For the last analyses, we level of curiosity on multiauthor papers. Females preserve excessive-location authorship positions in 60.6% of the a lot of-author papers in the start science literature, in contrast with 57.9% in the reproducibility literature. Disguise that with gender parity, the expected percentage of a lot of-author papers with a girl in a excessive-location (first or last) author situation would be 75% (produced from a 25% likelihood of girl first and last, a 25% likelihood of girl-first and man-last, and a 25% likelihood of man-first and girl-last).

We performed a regression evaluation to better label gender differences in excessive-location authorship positions across the 2 literatures. Particularly, we fit a logistic spline regression mannequin controlling for time traits, team size, and manuscript style (i.e., journal article or convention continuing). For this evaluation, we extinct a subset of multiauthored papers for which we had been ready to enact whether or no longer or no longer a girl holds a excessive-location situation (i.e., where with some stage of self belief, the gender of the first and last author is seemingly to be nice, or the gender of the first or last author is seemingly to be identified as feminine despite the real fact that the others couldn’t be identified). We additionally excluded 28 start science papers and 40 reproducibility papers with more than 12 authors to manual nice of giving these papers disproportionate have an effect on on regression fit. The resulting dataset consisted of 454 start science papers and 955 reproducibility papers. After controlling for team size, year of publication, and manuscript style, we chanced on that multiauthor papers in the reproducibility literature have 61% decrease odds of getting a girl in a excessive-location authorship situation in contrast with the start science literature (P < 0.001; SI Appendix, Desk S3). Thus, whereas ladies folk are underrepresented in excessive-location author positions on multiauthored papers in each literatures (relative to gender parity), there would possibly be considerably greater representation of girls folk authors in excessive-location author positions in the start science (vs. reproducibility) literature.

On the change hand, every other time, every other hypothesis is that these gender differences in excessive-location author positions are merely driven by disciplinary field. To seek this possibility, we fit the mannequin controlling for the academic field of seek and chanced on identical effects (see SI Appendix for particulars). Thus, the gender representation distinction in excessive-location authorship positions in start science (vs. reproducibility) is well-known to disciplinary field.

Females’s excessive-location authorship is more constrained by team size in reproducibility than in start science.

Females’s excessive-location authorship is another way patterned by team size in these literatures. Within multiauthored papers, ladies folk’s likelihood of authoring in excessive-location positions in the start science literature is better in smaller groups (two- to some-author papers; Fig. 4) and remains slightly constant as groups change into greater (Fig. 5, Left). On the change hand, at some level of the reproducibility literature, ladies folk are much less seemingly to author in excessive-location positions in smaller groups (two- to some-author papers) and sure to enact so in greater groups (six- to seven-author papers). Regression analyses verify this distinction after controlling for assorted necessary variables, in conjunction with publication year and manuscript style (Fig. 5, Left).

Fig. 4.

Crew size and girls folk’s representation in excessive-location positions in multiauthor papers. Females’s representation in excessive-location authorship positions (first and last authorship) is patterned another way by team size in the start science and reproducibility literatures. Females delight in excessive-location positions continuously across smaller and greater groups in start science, whereas they enact so more steadily in greater groups in the reproducibility literature.

Fig. 5.

Estimated regression effects of team size and year of publication on ladies folk’s representation in excessive-location positions in multiauthor papers. (A) Females participation and team size. Females have greater rates of excessive-location authorship in greater groups within reproducibility, whereas rates are comparatively and continuously excessive in start science across team sizes. (B) Females’s participation over time. In start science, the representation of girls folk in excessive-location positions has grown over time, whereas in reproducibility, it has declined. Values are logistic regression estimates shown on the prospect scale, with 95% CIs indicated in grey. To supply the estimates, the x-axis variable and literature class are diversified, whereas the last mannequin variables are mounted (see Concepts for particulars).

We additionally thought-referring to the change hypothesis that field differences is seemingly to be driving the noticed relationship between ladies folk’s participation and team size. To seek this, we performed the equivalent regression analyses stratified by field and chanced on that the outcomes had been largely well-known across fields. That is, ladies folk are underrepresented in excessive-location author positions on smaller groups in the reproducibility literature (in contrast with the start science literature; see SI Appendix for a detailed description of these analyses and findings). Taken together, we uncover that ladies folk’s participation in excessive-location author positions is more constrained in reproducibility than in start science and occurs more steadily in greater groups at some level of the reproducibility literature.

Females’s representation in excessive-location author positions is rising in start science over time and lowering in reproducibility. Extra regression analyses indicate that in the start science literature, the representation of girls folk in excessive-location authorship positions has grown over time, whereas it has declined or failed to lengthen in the reproducibility literature. We uncover that the possibilities of a girl preserving a excessive-location situation in the start science literature has grown at a charge of ∼15.6% (P < 0.01) year-over-year from 2010 to 2017 (SI Appendix, Desk S3), controlling for team size and manuscript style. Within the reproducibility literature, over the equivalent time frame the representation of girls folk in excessive-location positions has declined at an estimated charge of ∼3.6%, even supposing this decline is no longer statistically necessary (P = 0.20). Inspecting the variation between these slopes unearths a statistically necessary distinction between ladies folk’s representation over time between these literatures (P < 0.01). Fig. 5, Dazzling illustrates the variation in traits over time between the 2 literatures on the prospect scale.

At last, we every other time explored the change field hypothesis: that ladies folk’s participation over time used to be driven by field differences. Particularly, we performed the equivalent regression analyses stratified by field and chanced on that the outcomes had been largely well-known across fields. That is, we chanced on increasing participation of girls folk in start science over time and lowering participation of girls folk in reproducibility in each field as an alternative of psychology, where ladies folk’s participation has grown over time in reproducibility (53) (see SI Appendix for detailed field analyses and findings).

Dialogue

Our outcomes indicate that the movement to beef up science includes two slightly self reliant groups of investigators with differing approaches: 1) start science and a pair of) reproducibility. These literatures have slightly few same old papers and authors, indicating they’re nice, nonoverlapping communities. Every presentations a considerably assorted neighborhood structure of how authors make contributions to particular particular person papers. Whereas the start science literature is considerably more interconnected with appreciate to coauthorship, the reproducibility literature is more fragmented. But one more indicator of these assorted emergent cultures comes from the semantic text evaluation, which means that the nature of explicitly prosocial cultures in the start science and reproducibility literatures range. Open science abstracts consist of more explicitly communal and prosocial phrases than enact reproducibility abstracts. Cohering with these structural and cultural divergences, we uncover assorted patterns of participation by ladies folk scholars. General, ladies folk are seemingly to buy management positions (i.e., excessive-location author positions) in the start science literature than in the reproducibility literature, and this greater participation is extra patterned by team size and time. When authorship groups are slightly tiny (e.g., two to some authors), ladies folk’s likelihood of authoring in excessive-location authorship positions is greater in start science in contrast with reproducibility. Females’s participation is more constrained in reproducibility—occurring more customarily in greater groups—whereas it is freer in start science (occurring as steadily in smaller and greater groups). At last, ladies folk’s participation in these literatures yields assorted temporal patterns as effectively, with rising participation in start science however lowering in reproducibility.

Given these findings, we argue that there are well-known causes for science customarily—in conjunction with each subcultures of science reform—to undertake inclusive and prosocial cultures. First, a convention that portrays science as noncommunal does no longer reflect how scientific work the truth is unfolds—particularly with in the present day’s emphasis on unparalleled challenges, transdisciplinary investigations, and network science. Certainly the (unfounded) prototype of a scientist is one in which a particular person scientist (customarily a white male) toils away alone in his laboratory except a flash of perception occurs in a “eureka!” moment (5456). This tradition is epitomized by about a of our most prestigious awards which have an even time particular particular person efforts and contributions over that of groups (e.g., Nobel prize, MacArthur Fellowship Award, NIH Director’s Pioneer Award, NSF Occupation Award; NIH “self reliant investigator” categorization). Furthermore, college evaluations for tenure and promotion continue to prize particular particular person efficiency nearly exclusively—in some cases requiring scientists to show their self reliant contribution to collaborative initiatives and/or calculating the selection of first- or last-authored (vs. coauthored) publications (57). This day’s science relies on groups coordinating their efforts to fragment insights and strategies, compose on previous work, and manufacture unique questions and approaches (58). These collaborative and complementary processes happen in the neighborhood (e.g., state work with assorted laboratories) as effectively as globally (e.g., broadening the scientific neighborhood, sharing equipment, recordsdata, and get entry to) (59). Science in the present day is seemingly to be a collaborative, than particular particular person, endeavor—where team size can subject. Certainly, greater and more various groups is seemingly to be essential to label greater impact (60). A local, on the opposite hand, is that whereas science is more and more team-essentially essentially essentially based, homophily processes mean that many groups are seemingly to be slightly homogeneous with regard to sociodemographic, behavioral, and intrapersonal traits (61). Consideration wishes to be paid, proactively, to the composition of groups.

Second, and per the level above, there would possibly be an rising appreciation amongst scientists and funding agencies that multidisciplinary “team science” is required to style out the most urgent scientific, social, and health concerns of our instances. Over the last decade, organizations in conjunction with NIH, NSF, and others have dedicated resources to facilitating team science. This work is evidenced by interdisciplinary and multidisciplinary team requirements in federal funding announcements and packages (e.g., Nationwide Institute of Frequent Medical Sciences Collaborative Program Grant for Multidisciplinary Teams, NSF Pickle of business of Multidisciplinary Activities, NIH Interdisciplinary Program in the Frequent Fund and its predecessors in the NIH Roadmap, the Nationwide Cancer Institute’s Science of Crew Science Toolkit, NSF Gigantic Files Regional Innovation Hubs Program, NSF Collaborative Computational Neuroscience Program, NSF Pickle of business of Multidisciplinary Activities) and loads assorted packages under the NSF and NIH roadmaps and priorities). Furthermore, funders are actively trying to address the underrepresentation of girls folk and minorities (e.g., NSF Broadening Participation), even supposing there are aloof inequities in these processes (62).

Certainly, the complexity of the concerns we’re the truth is facing in science demands the abilities of a lot of disciplines working in coordinated style (63, 64). For instance, addressing the affirm of opioid addiction requires the integrated recordsdata of researchers who focus on distress, addiction, neuroscience, economics, computer science, psychology, sociology, biochemistry, demography, medication, and public health, upright to call about a. Intellectually various, multidisciplinary groups create unique insights by combining existing recordsdata in modern methods (65, 66). Truly, recordsdata from the US Patent and Emblems Pickle of business show that patents generated by groups represented more breakthroughs, touchdown amongst the wreck 95% of all cited patents, than those from lone inventors, suggesting their generative nature (67). Equally, multiauthored articles are more customarily cited than single-authored articles (60, 68, 69), and whereas some have argued that this is seemingly to be due to self-citation, others have instantaneous that it is more seemingly that extremely collaborative initiatives consist of more various recordsdata and greater quality tips, which wreck in greater impact (70). Importantly, it has additionally been instantaneous that whereas trim groups advance science and abilities, tiny groups can disrupt the established scientific belief. Each forms of contributions appear like of elementary importance (71, 72). Finally, if various team science is the future, institutions must reconsider for my fragment constructed incentive structures as these structures couldn’t promote rapid progress if scientists remain tied to particular particular person incentives.

At last, a third purpose to use a prosocial scientific tradition, per our findings and that of assorted learn, is that noncommunal practices and values would possibly almost certainly deter individuals that cost communal, interdependent, and prosocial objectives, in conjunction with ladies folk (14), underrepresented minorities (41, 73), first-generation college students (73), and communally oriented men (14). If the movement to beef up science is to harness this diversity, the start science level of curiosity currently appears to be like to be more welcoming and inclusive than reproducibility. On the change hand, each foci have the conventional purpose of bettering our recordsdata, rigor, and belief. These contributions are seemingly enhanced when a various range of scientists are completely taking part in either scheme’s efforts.

Lack of Diversity Can Be Problematic for Science.

Lack of social diversity (e.g., gender and racial diversity) within scientific groups is seemingly to be detrimental to science. There are many case learn where homogenous groups have produced serious failures of recordsdata with regard to serious outcomes. For instance, with out a ladies folk on engineering and vogue groups, coronary heart valves and seat belts are made that easiest fit men’s bodies (considerably rising mortality rates for ladies folk) (74), converse-recognition system easiest recognizes the voices of guys (74), and image-recognition system tags Dusky individuals as apes (75). Including and heeding the voices and experiences of a range of oldsters can foster outcomes that support a a lot broader range of oldsters. Whereas groups with more gender and cultural diversity are seemingly to manufacture unique merchandise and introduce radical innovations to market (76, 77), and whereas papers authored by various scientific groups have more citations and greater impact factors (78), the mere presence of social diversity is no longer continuously adequate to foster equal participation of various social groups. For instance, a trim-scale evaluation of up-to-the-minute scientific articles chanced on that ladies folk had been considerably seemingly to be linked to technical initiatives, whereas men had been linked to conceptual initiatives (79). Equally, in gender-various engineering groups of students, ladies folk had been underrepresented in presenting technical disclose material, whereas men had been overrepresented (80). Certainly, the capability of social diversity customarily goes untapped, resulting in null or adverse outcomes on neighborhood efficiency (8184).

To capitalize on the capability of social diversity, groups want to straight address the challenges that can accompany social diversity. For instance, interactions and communique within various groups is seemingly to be more refined, particularly in the foundation (8587). On the change hand, there would possibly be colossal seemingly of social diversity, particularly in complicated initiatives. Socially various groups encode and task recordsdata more precisely (88), particularly when the sharing of disparate info is a requirement for fulfillment (89). The mere presence of oldsters from socially various backgrounds alters the cognition and behavior of majority neighborhood members to foster improved and proper thinking and communique (90). Within the presence of social diversity, majority neighborhood members elevate more info and manufacture fewer merely errors, and when errors are made, they’re seemingly to be corrected (90). When questions and dissent are raised in socially various groups, it provokes more thought and consideration than when the categorical same concerns are raised in homogenous groups (91). At last, the presence of underrepresented neighborhood members can foster greater participation from assorted underrepresented neighborhood members. One instance is that gender-various groups with more ladies folk foster ladies folk’s lively participation in team initiatives, whereas groups that are produced from mostly men customarily render ladies folk still (86).

The Rising Movements to Toughen Science.

The psychological and brain sciences (PBS) are at the forefront of efforts to redefine the principles and requirements of science (92, 93). There is a lot to be taught from this rising movement, and several other assorted fields (9498) are equally taking stock, in conjunction with biostatistics (99, 100), computer science (101), and medication (102, 103). For instance, the team science components to bettering science is seemingly to be noticed in theoretical and experimental physics where investigative necessity has promoted trim-scale consortia and a hit gadgets of scientific collaboration (104). Equally, the collaborative discipline of structural biology established requirements for sharing and deposition of code and recordsdata (see Collaborative Computational Mission No. 4 and Be taught Collaboratory for Structural Bioinformatics), and these communal practices coincided with a broader participation of girls folk in the field over its ten an extended time (105).

In sum, start science has the seed of a communal and sharing tradition that, if cultivated, would possibly almost certainly continue to foster the inclusion and participation of girls folk. We counsel that pivoting towards this cultural vogue would possibly almost certainly support to diversify the reproducibility movement without detracting from its core objectives. We imagine that the collaborative, forward-having a see aspect of start science has the capability to facilitate diversity and inclusiveness in two methods. First, the sharing of code, recordsdata, and resources lowers the boundaries and entry cost to take half in science, thus establishing a more equal playing field and bettering the inclusion of underrepresented groups—as an illustration, scientists working in minority-serving institutions with much less get entry to to funding and various resources (106). Second, a convention of sharing, interdependence, and collaboration is per learn (cited above) that implies these cultural aspects are more exquisite to ladies folk, individuals of coloration, individuals from decrease socioeconomic backgrounds, and communally oriented men.

Some aspects of the movements to beef up science have explicitly centered on cultural values and practices to promote inclusivity. For instance, the Society for the Improvement of Psychological Science explicitly entails working towards an inclusive tradition in its mission observation, and the net methods and practices discussion neighborhood PsychMAP used to be founded to produce a more collaborative and communal dwelling for discussion (see neighborhood ground principles). To make definite that, reflecting and finding out from within a cultural shift is refined. The evaluation we offer right here means that we can aloof enact more to beef up science by scheme of social diversity. We indicate that the advantages of team science will seemingly be realized when such groups are each socially and intellectually various and operate in contexts that welcome and pursue diversity, so that innovation, creativity, and the quality of science can flourish—despite an initial interval of adjustment and discomfort. Science wants the participation of girls folk and various underrepresented groups. The objectives and beliefs of start science have the capability to promote diversity and broader scientific participation. On the change hand, the promise of these rising cultural traits is no longer yet a nice wager; indeed, some aspects of the dominant scientific tradition can deter participation amongst the very participants who would possibly almost certainly make contributions to the strength of various thinking. By fostering cultural switch towards prosocial values, sharing, education, and immoral-disciplinary cooperation, moderately than independence and competitiveness, the movement to beef up science would possibly almost certainly consequence in greater recordsdata generation, democratization, and inclusiveness in science.

Specific steps can and are being made to facilitate and advance the variety we’re promoting. Departments, institutions, and professional societies can create communal and prosocial structures for start science, such as start infrastructure and initiatives to permit for establishing academic networks, practising, resources, and recordsdata sharing. Other particular examples consist of the advance of Transparency and Openness Promotion Pointers (39) and the establishment of cloud-essentially essentially essentially based platforms and associated particular person communities for learn asset sharing. Look examples in PBS, recordsdata in OpenNeuro.org (107), analyses in brainlife.io (108110), and seek registrations in Open Science Framework (39). Particular particular person researchers can be taught referring to the who, when, how, and why of their groups, in conjunction with attending to the variety of oldsters represented, figuring out alternatives to consist of various voices, and examining causes and bounds for groups’ or participants’ participation. Organizations that highlight the collaborative and communal aspects of scientific processes and success can characteristic connections in science, acknowledging how others support overcome barriers and rewarding groups that embody the values of start science. Every researcher can work towards broadening their collaboration and mentoring networks. We aid readers and all members of the scientific neighborhood to embrace a finding out mindset referring to team science and socially various groups. Science continuously has more to coach, and the rewards of a cultural shift are no longer free; they reach from investments of time, vitality, belief, and action.

Concepts

Files Sources.

A total of 11,338 long-established papers had been still the use of the snapshot of MAG (https://academic.microsoft.com) on February 23, 2018. To safe the datasets, we searched MAG for all publications with particular “field of seek tags” as “start science” or “reproducibility.” The sphere of seek tags are produced by an internal Microsoft algorithm essentially essentially essentially based on the contents and metadata (e.g., abstracts) of every paper (no longer author-generated; see ref. 111 for particulars). Amongst all the records, easiest 68 papers had been labeled as each “start science” and “reproducibility”. Furthermore, of the 36,296 recurring author IDs represented in these literatures, completely about a (n = 457) have authored in each literatures. These findings counsel that the 2 literatures are increasing moderately independently. For the capabilities of our analyses, we eradicated papers that had been labeled as each “start science” and “reproducibility” to manual nice of double-counting papers and skewing analyses. Amongst the last records, we easiest thought-about formal revealed papers of the style “journal” or “convention.” The resulting dataset included 3,431 start science papers and 7,839 reproducibility papers.

Amongst the last records, we easiest thought-about formal revealed papers of the style “journal” or “convention” (document sorts “book,” “book chapter,” and “patent” had been eradicated). We additionally eradicated 43 papers with replica titles. We examined the last selection of papers revealed as soon as a year within each literature (SI Appendix, Fig. S1). As completely about a start science papers had been revealed forward of 2010, and few papers in either field had been revealed in 2018, we easiest use recordsdata for papers revealed between 2010 and 2017, which entails 2,926 papers in total, with 879 start science papers and a pair of,047 reproducibility papers. This is the closing dataset extinct for all analyses, as an alternative of where otherwise famed.

Files compiled for the analyses is seemingly to be chanced on at Open Science Framework (https://osf.io/97vcx) (112), and the code extinct for this work is in the market at GitHub (https://github.com/everyxs/openScience).

Per the sample between 2010 and 2017, we constructed the paper coauthorship networks for 879 start science papers and a pair of,047 reproducibility papers. Every node represents a scientific article. Two nodes fragment an edge if no decrease than one author appears to be like in each papers. Per MAG author IDs, we identified 3,157 recurring author names in the start science literature and 8,766 in the reproducibility literature. Within the start science literature, the network contains 389 edges (i.e., pairs of papers with out a decrease than one author in same old) and 856 edges in the reproducibility literature.

Community Prognosis.

For each networks, we performed an edge density and linked parts evaluation as follows.

Edge density.

For an undirected network with n nodes and m edges, the edge density is defined as:

ρ=m[n×(n1)]/2.

To test whether or no longer the start science network has greater edge density than the reproducibility network, we performed a one-sided Fisher’s proper test. We assumed a binomial edge generation task between all pairs of nodes and examined the hypothesis that the possibilities ratio of the 2 networks is greater than one. We estimated the possibilities ratio the use of the edge density of every networks,

ρ1(1ρ2)ρ2(1ρ1),

where

ρ1

represents the edge density of the start science network and

ρ2

the edge density of the reproducibility network. The percentages ratio test used to be extinct to deal with the tiny values of the network density (0.057 and zero.047%), against a test the use of a linear scale. The test rejects the null hypothesis that the start science network does no longer have greater edge density than the reproducibility network with a P cost of seven.35e−5.

Linked parts.

We performed an additional evaluation to estimate how linked (or isolated) the subcomponents of every network are. For an undirected network, a linked factor is defined as a maximal subgraph in which any two nodes are linked to each assorted by a series of edges. In our case, each networks are sparse with many separate linked parts. We in contrast the 2 networks in phrases of the size of the greatest linked factor, as effectively because the ACS, which is defined because the network size divided by the selection of linked parts. The linked parts evaluation is performed the use of the system Gephi (46).

As a robustness check, we performed the equivalent edge density and linked parts evaluation amongst the multiauthored papers easiest (as an alternative of single-authored papers). These analyses and visualizations is seemingly to be show in SI Appendix, Fig. S2.

Semantic Textual disclose material Prognosis of Abstracts.

Starting with the 2,926 papers from each start science and reproducibility described above, we first eradicated papers without in the market abstracts (205 start science and 815 reproducibility papers) after which eradicated those with non-English titles (79 start science and 63 reproducibility papers), as nice the use of the R textcat package (113). The resulting dataset extinct in the text evaluation consisted of 1,764 papers, in conjunction with 595 start science papers and 1,169 reproducibility papers. We then performed fashioned text preprocessing and eradicated stop words, stemming, and punctuation and converted the text to lowercase the use of the SentimentAnalysis R package. We measured prosocial constructs in the text by counting the frequency of occurrence of 127 words in a validated dictionary (113) (e.g., make contributions, aid, support, nurture; SI Appendix, Desk S2). This dictionary has been shown to have acceptable settlement with human judges (r = 0.67) (114). The prosocial word density is calculated because the ratio of the selection of prosocial words over the entire selection of words in each summary. Semantic text evaluation stratified by field is described in SI Appendix, Fig. S5.

Gender Participation Analyses.

We performed a aged gender (male, feminine) evaluation by figuring out the gender of the first and last authors given their name. To enact so, we extinct the gender R package (https://github.com/ropensci/gender) (115); to resolve the prospect of the first and last author to be a female. The gender package makes use of historical recordsdata on gender to foretell the gender of a particular person essentially essentially essentially based on their given name(s) and birth year or year range. For each paper, we assumed birth year to be such that the author would be between the ages of 25 and 65 at the time of publication. To title the first name of every author, we first identified the factor of every author name by assuming that every name factor used to be separated by one dwelling in the guidelines. We then thought-referring to the first and middle names (when in the market) and excluded all assorted initials to produce gender detection. We computed the prospect of being feminine for every author with out a decrease than one paunchy (noninitial) first or middle name portion. Authors with likelihood over 0.5 had been labeled “feminine” and participants with likelihood under 0.5 had been labeled “male.” We extinct the “ssa” possibility of the gender package, which appears to be like to be like up names essentially essentially essentially based from the US Social Security Administration child name recordsdata from the interval 1932 to 2012.

For Figs. 3 and 4, we labeled papers as having a girl in a excessive-location author situation if either the first or last author used to be labeled “feminine” the use of the scheme described above. We excluded papers with unknown excessive-location feminine authorship, which entails papers with each the first and last author labeled “unknown” and papers with one situation “male” and the various “unknown.” We excluded single-author papers, since a decrease percentage of those would be expected to have feminine excessive-location authorship in contrast with multiauthor papers (since in a probabilistic sense there are two “probabilities” to manufacture excessive-location authorship in multiauthor papers however easiest one “likelihood” in single-author papers). In Fig. 3, we additionally excluded papers with more than 15 authors for the sake of visualization.

For Fig. 4, we performed logistic regression evaluation to quantify how the rates of girls folk’s excessive-location authorship in multiauthor papers diversified by team size within each literature. We included a spline term for team size within each literature, given the proof for a nonlinear relationship between team size and rates of feminine lead authorship. We excluded 28 start science papers and 40 reproducibility papers with more than 12 authors to manual nice of undue have an effect on on the estimation of these spline phrases. The resulting dataset consisted of 454 start science papers and 955 reproducibility papers.

Particularly, we fit a logistic regression mannequin pertaining to the log-odds of getting a girl in a excessive-location author situation to the year of publication, the selection of authors (the use of a versatile spline term), the kind of publication (convention proceedings or journal article), and the literature to which every paper belongs. We allowed the outcomes of year of publication and selection of authors to be nice one by one for every literature by scheme of interaction phrases. We estimated the mannequin coefficients the use of the R gam function from the mgcv package the use of a binomial family with logit link. This function represents quiet coefficient curves as penalized splines and makes use of generalized immoral-validation to estimate the smoothness of every curve (116). Particularly, we fit the mannequin

log{Pr(Yi=1)1Pr(Yi=1)}=β0+β1Repi+β2Yeari+β3YeariRepi+f1(Authorsi)+f2(AuthorsiRepi)+β4Confi+ϵiϵiN(0,σ2)

where

Yi=1

if paper i has a girl in a excessive-location author situation,

Repi=1

if paper i belongs to the reproducibility literature,

Yeari

is the year of publication (centered at 2017),

Authorsi

is the team size (centered at 2, the minimum cost for multiauthor papers), and

Confi=1

if the paper is a convention continuing. The capabilities

f1()

and

f2()

are quiet coefficient curves that plan team size to the log-odds of getting a girl excessive-location author in each literature, given mounted values of the various coefficients.

Per the estimated regression coefficients and SEs, we estimated the log-odds of getting a girl in a lead authorship situation given particular gadgets of predictor variables, alongside with same old 95% CIs. We then transformed the log-odds and CIs to odds and probabilities for better interpretability. SI Appendix, Desk S3 experiences estimates and CIs on the possibilities scale for every parametric (i.e., nonspline) coefficient. Briefly, we uncover that the fabricate of belonging to the reproducibility literature is adverse with an estimate of 0.393, representing ∼61% diminished odds of getting a girl in a excessive-location situation in contrast with papers in start science for a given team size, year of publication and manuscript style. The fabricate of later publication year is nice for start science papers however adverse for reproducibility papers. All parametric coefficients are statistically necessary at the 0.05 stage.

The effects of publication year and team size are explored in extra part by examining the anticipated probabilities of getting a girl in a excessive-location situation as year and team size range. Fig. 5 depicts the estimates and 95% CIs for the prospect of getting a female in a excessive-location situation for assorted values of these variables. CIs on the prospect scale are constructed by making use of the inverse logit transformation [i.e.,

p(x)=exp{x/(1x)}

] to the conventional 95% CI on the logit scale. The estimates and CIs subsequently symbolize predicted probabilities. In Fig. 5, Left, we fix Convention = FALSE and Yr = 2017, whereas allowing team size to range for every literature. The outcomes show that for start science papers, there is a adverse fabricate of team size, with smaller groups having slightly greater likelihood of getting a girl in a excessive-location situation. For reproducibility papers, there is a nonlinear fabricate of team size, with the prospect of getting a girl in a excessive-location situation being markedly decrease for tiny groups and peaking for groups with roughly seven authors forward of declining slightly. In Fig. 5, Dazzling, we fix Convention = FALSE and Crew Size = 4 (stop to the mean cost), whereas allowing the year of publication to range for every literature. We gape a inserting distinction in the fabricate of year of publication for start science and reproducibility papers, with an rising vogue over time for start science papers and a moderately lowering vogue over time for reproducibility papers. This implies rising participation of girls folk in excessive-location positions at some level of the start science literature over time and a decline or stagnation in the reproducibility literature. Robustness checks controlling for and stratifying analyses by field are provided in SI Appendix, Desk S4 and Figs. S3 and S4.

Acknowledgments

M.C.M. used to be supported by NSF CAREER Grant DRL-1450755 and NSF Grant HRD-1661004 and by Russell Story Foundation Grant 87-15-02. A.B.D. used to be supported by NSF Grant GSE-1232364. F.P. used to be supported by NSF Grants OAC-1916518, IIS-1912270, IIS-1636893, and BCS-1734853; NIH Grant 1R01EB029272-01; a Google Cloud Be taught Award; a Microsoft Investigator Fellowship; the Indiana University Areas of Emergent Be taught initiative “Learning: Brains, Machines, Younger individuals”; and the Indiana University Pervasive Technology Institute. We acknowledge Angela Sharpe and Cassidy Sugimoto for his or her thoughtful discussion referring to the subject issues in the paper, which had been mirrored in the manuscript, and Michael Jackson for support with the kind of the figures.

Footnotes

  • Creator contributions: M.C.M., A.F.M., J.M., X.Y., P.L.M., S.R., A.B.D., and F.P. designed learn; M.C.M., A.F.M., J.M., X.Y., P.L.M., S.R., A.B.D., and F.P. performed learn; A.F.M., J.M., and X.Y. analyzed recordsdata; M.C.M., A.F.M., J.M., X.Y., S.C., N.D., M.D., S.A.F., J.A.G., E.L.H., J.M.H., A.L., C.A.M.-R., L.E.P., S.P.P., Okay.A.R., A.R., D.T.S., Okay.S., D.S., J.L.S., V.J.T., D.B.T., D.A.W., P.L.M., S.R., A.B.D., and F.P. wrote the paper.

  • The authors repeat no competing curiosity.

  • This text is a PNAS Relate Submission.

  • *This day, it is acknowledged that reproducibility can have assorted meanings in assorted fields of science (29–31). We explored how assorted approaches to reproducibility (e.g., repeatability, recordsdata sharing) had been labeled by our task. We chanced on that every person papers with the MAG field of seek label “repeatability” had been labeled by our scheme as “reproducibility” papers—in step with the Nationwide Academy of Sciences (NAS) conceptualization of reproducibility (29). Furthermore, nearly all papers with the MAG field of seek tags “start recordsdata” or “recordsdata sharing” had been labeled by our scheme as “start science” papers, as intended (SI Appendix, Desk S1). We would possibly almost certainly aloof additionally show that the dataset for this narrative used to be compiled in 2018 (SI Appendix)—1 y forward of the distinction between reproducibility and replicability used to be formalized by the NAS narrative (29).

  • This text contains supporting recordsdata online at https://www.pnas.org/look up/suppl/doi: 10.1073/pnas.1921320117/-/DCSupplemental.

Leer Abstract