Abstract
The widely held assumption that any important scientific information would be available in English underlies the underuse of non-English-language science across disciplines. However, non-English-language science is expected to bring unique and valuable scientific information, especially in disciplines where the evidence is patchy, and for emergent issues where synthesising available evidence is an urgent challenge. Yet such contribution of non-English-language science to scientific communities and the application of science is rarely quantified. Here we show that non-English-language studies provide crucial evidence for informing global biodiversity conservation. By screening 419,680 peer-reviewed papers in 16 languages, we identified 1,234 non-English-language studies providing evidence on the effectiveness of biodiversity conservation interventions, compared to 4,412 English-language studies identified with the same criteria. Relevant non-English-language studies are being published at an increasing rate, and can expand the geographical (by 12-25%) and taxonomic (by 5-32%) coverage of English-language evidence, especially in biodiverse regions, albeit often based on less robust study designs. Our results show that synthesising non-English-language studies is key to overcoming the widespread lack of local, context-dependent evidence and facilitating evidence-based conservation globally. We urge wider disciplines to rigorously reassess the untapped potential of non-English-language science in informing decisions to address other global challenges.
Introduction
History demonstrates that important scientific information is published not just in English but also other languages. The structure of the Nobel Prize-winning antimalarial drug was first published in Chinese [1]. An important rule regarding biodiversity was founded on evidence published in Spanish [2]. Many of the earliest papers on COVID-19 were written, again, in Chinese [3]. Yet the contribution of such non-English-language science to scientific communities, and the broader society, is rarely quantified.
We test this untapped potential of non-English-language science through an assessment of non-English-language studies’ contribution to evidence synthesis—the process of compiling and summarising scientific information from a range of sources. Evidence synthesis plays a major role in informing decisions for tackling global challenges in fields such as healthcare [4], international development [5], and biodiversity conservation [6]. To date non-English-language studies have largely been ignored in evidence synthesis [7–9]. The consequences of this common practice are, however, rarely investigated in most disciplines apart from healthcare. And even there, the focus has almost exclusively been on how including non-English-language studies might change the statistical results of meta-analyses [10, 11] (see Supplementary Text for a review of earlier relevant studies). However, non-English-language studies may also enhance the synthesis of evidence with specific types of scientific information that is not available in English-language studies, especially in disciplines dealing with more geographically and taxonomically diverse targets and phenomena than healthcare [12].
Synthesising non-English-language studies could be an effective avenue for reducing the existing, severe gaps in the geographical and taxonomic coverage of available scientific evidence for biodiversity conservation [13, 14]. Compiling evidence on what does or does not work in biodiversity conservation, and informing decisions with robust scientific evidence is critical to halting the ongoing biodiversity crisis [6]. As local and context-dependent evidence is crucially required for conservation-related decision making [15], the geographical and taxonomic gaps in evidence, especially in biodiverse regions, pose a major challenge to our scientific understanding of the biodiversity crisis and the implementation of evidence-based conservation globally. Non-English-language studies could be particularly important in biodiversity conservation for the following reasons. First, over one-third of scientific documents on biodiversity conservation are published in languages other than English [16]. Second, gaps in globally compiled English-language evidence are often found in areas where English is not widely spoken [13]. Third, important evidence in biodiversity conservation is routinely generated by local practitioners, who often prefer publishing their work in their first language, which for many is not English [16].
Here we adopted the discipline-wide literature search method [17] to screen 419,680 peer-reviewed papers in 326 journals, published in 16 languages (Data S1), to identify non-English-language studies testing the effectiveness of interventions in biodiversity conservation (see Materials and Methods). Combining this dataset with English-language studies identified with the same criteria, stored in the Conservation Evidence database [17], enabled us to assess the contribution of non-English-language studies to evidence synthesis through the testing of the following common perceptions that are rarely corroborated together: (i) the amount of relevant scientific evidence that is available only in non-English languages is negligible [18], (ii) the number of relevant studies being published in non-English languages has been decreasing over time [19], (iii) the quality of non-English-language studies (measured using the study designs adopted) is lower than that of English-language studies [7], and (iv) evidence published in English represents a random subset of evidence published across all languages [12].
Results
Our search elicited a total of 1,234 eligible non-English-language studies (including 53 studies on amphibians, 247 on birds, and 161 on mammals, which were used for a detailed species-level comparison with English-language studies) testing the effectiveness of conservation interventions, published in 16 languages (Data S2 and S3). This adds a considerable amount of scientific evidence for biodiversity conservation to the Conservation Evidence database, which now stores 4,412 English-language studies (including 284 studies on amphibians, 1,115 on birds, and 1,154 on mammals). The proportion of eligible studies in each journal varied among languages, with Japanese (the highest proportion of eligible studies in a journal was 26.7%), Hungarian (15.3%), French (12.9%), and German (9.1%) showing particularly high proportions (largely < 5% of the studies screened were eligible in journals of other languages) (Fig. S1). In all languages, except Hungarian, many journals searched had almost no eligible studies, showing that our search had covered and gone beyond most of the relevant journals (see Limitations in Materials and Methods for more details).
The yearly number of eligible non-English-language studies published in each journal has increased significantly over time, especially since 2000, in six out of the 12 languages covered, with Portuguese and Russian showing a particularly rapid increase, while traditional Chinese also showed a marginally significant increase (Fig. 1). The other five languages did not show a significant change in the number of eligible studies over time. This result thus refutes the common perception that the number of non-English-language studies providing evidence is declining. The recent increase in eligible studies indicates that performing searches only on volumes from the most recent ten years in some long-running journals had minimal impact.
Our results largely support one of the common perceptions—that non-English-language studies tend to be based on less robust study designs. Studies in ten out of the 16 languages we searched were significantly more likely to adopt less robust designs, compared to English-language studies, when controlling for the effect of study taxa and countries where English-language studies were conducted (Fig. 2 and Table S1). Of the other six languages showing no significant difference in designs from English-language studies (Persian, Portuguese, Spanish, traditional Chinese, Turkish, and Ukrainian), only Portuguese and Spanish had reasonable sample sizes (i.e., ten or more studies in each taxonomic group), indicating that designs adopted in studies in those two languages were comparable to those in English-language studies.
There was a clear bias in study locations between languages. English-language studies were conducted in a total of 952 of the 2° × 2° grid cells and non-English-language studies in 353 grid cells, 238 of which had no English-language studies (those grid cells shown in black in Fig. 3). Therefore, non-English-language studies expanded the geographical coverage of English-language studies by 25%. More non-English-language studies tended to be found in grid cells with fewer English-language studies, especially in East/Central/Western Asia, Russia, northern Africa and Latin America (Fig. 3 and Fig. S2), but the relationship was not significant when controlling for spatial autocorrelation (posterior median slope in a conditional autoregressive model: -0.012, 95% credible interval (CI): -0.032 – 0.005; see an inset in Fig. 3). Non-English-language studies expanded the geographical coverage based on English-language studies by 12% for amphibians (Fig. S3), 16% for birds (Fig. S4), and 12% for mammals (Fig. S5). In all three taxa, significantly more non-English-language studies were found in grid cells with fewer English-language studies (amphibians: slope: -0.51, 95% CI: -0.94 – -0.17; birds: slope: -0.23, 95% CI: -0.44 – -0.073; mammals: slope: -0.48, 95% CI: -0.74 – -0.25; also see insets in figs. S3-5).
The 1,234 non-English-language studies together provided evidence on the effectiveness of conservation interventions for a total of 1,954 unique species recognised by the International Union for Conservation of Nature (IUCN) (including 40 amphibian, 564 bird, and 194 mammal species). Although species with more studies in non-English languages also tended to have more studies in English for all three taxa (generalised linear mixed models for amphibians: slope = 0.12, z = 7.93, p < 0.001; birds: slope = 0.060, z = 13.18, p < 0.001; mammals: slope = 0.026, z = 5.65, p < 0.001; also see insets in Fig. 4), non-English-language studies provided scientific evidence on the effectiveness of conservation interventions for an additional nine amphibian, 217 bird, and 64 mammal species that were not covered by English-language studies (Fig. 4), meaning 5%, 32%, and 9% increases in the evidence coverage of amphibian, bird, and mammal species, respectively. Similarly, non-English-language studies increased the evidence coverage of threatened species (Critically Endangered, Endangered and Vulnerable species classified in the IUCN Red List of Threatened Species) by 23% for birds and 3% for mammals. All threatened amphibian species covered by non-English-language studies were also studied in English-language studies (Fig. S6). Threatened species with more studies in non-English languages had fewer studies in English for birds (slope = -0.34, z = -2.35, p = 0.019) but not for mammals (slope = 0.030, z = 0.923, p = 0.356; also see insets in Figure S6. Threatened amphibians could not be modelled as only two species were covered by non-English-language studies).
Discussion
Our analyses demonstrate that three out of the four common perceptions on the role of non-English-language scientific knowledge are not supported by evidence. We show that, instead, (i) a considerable amount of scientific evidence underpinning effective conservation is available in non-English languages (over 1,000 studies found in our searches), (ii) the number of published studies providing such evidence has been increasing in many languages, and (iii) non-English-language studies can provide evidence that is relevant to species (including threatened species) and locations (including highly biodiverse regions, such as Latin America) for which little or no English-language evidence is available. These results, based on a global empirical analysis of 5,646 studies in 17 languages, corroborate earlier arguments on the potential importance of non-English-language scientific knowledge in evidence-based biodiversity conservation [16]. A poor availability of species- and location-specific evidence, especially in countries where English is not widely spoken, has been recognised as a major impediment to evidence-based conservation [20], as scientific knowledge is often used only if it is relevant to the specific context of policies and practices [15, 21]. Meanwhile, a systematic bias in study characteristics, such as species and ecosystems studied, has been found between English- and Japanese-language studies in ecology [12]. Our study attests to the between-language bias in study characteristics, namely study species and locations, at the global scale, showing that incorporating non-English-language studies in evidence syntheses is an effective approach to rectifying biases and filling gaps in the availability of evidence over space and species. Examples of such non-English-language evidence on threatened species include a Spanish-language study testing the use of guardian dogs to alleviate conflicts between low-income livestock farmers in northern Patagonia and carnivores including endangered Andean mountain cats (Leopardus jacobita) [22], and a Japanese-language study reporting the effectiveness of relocation for endangered Blakiston’s fish owls (Bubo blakistoni) [23].
The other perception, that non-English-language studies tend to adopt less robust designs, seems to be supported by our results, although a reasonable number of non-English-language studies with robust designs also exist, especially in Portuguese (25 studies with Randomised Controlled Trial) and Spanish (13 studies with Before-After-Control-Impact and three with Randomised Controlled Trial). Scientific evidence presented in non-English-language studies could thus be lower in quality, and suffer from more serious biases, on average, compared to that provided by English-language studies [24]. This difference in evidence quality between English-language and non-English language studies is likely to create a trade-off in evidence-poor regions, between the availability of context-specific evidence and the quality of evidence; for some species and locations, the only available evidence might be found in non-English-language studies based on less robust designs [25]. Nevertheless, blindly discarding such lower-quality, yet relevant, studies—a common practice in conventional evidence syntheses—could unnecessarily delay, misinform, or hinder evidence-based decision making, especially in disciplines, such as conservation, where robust evidence bases are patchy [14], and for emergent issues, such as pandemics, where making the best use of available evidence is an urgent challenge [26, 27]. A promising approach here is the model-based synthesis of evidence with varying qualities and degrees of relevance to a specific context [24], where non-English-language studies are expected to play a crucial role as an important source of highly context-specific evidence.
We should note, however, that even searching for, and including, non-English-language studies would not fully address the large evidence gaps in some regions faced with the most pressing issues including biodiversity loss, such as Southeast Asia, tropical Africa and Latin America. Therefore, the use of existing non-English-language science is not a panacea. Generating more local evidence, based on robust study designs, and publishing it in any language should be further encouraged and supported globally, but especially in those evidence-poor regions, for example, through the distribution of free teaching materials to facilitate the testing of conservation interventions [28] (also see Limitations in Materials and Methods for other limitations).
This study showcases the continued vital role of non-English-language studies in providing evidence for tackling the ongoing biodiversity crisis, given the increasing number of relevant studies being published in many non-English languages. However, the degree of importance of such evidence will vary depending on the topic and discipline of focus. Relatively little evidence may be available in non-English languages for a highly specific purpose—for example for understanding the effectiveness of a single intervention for a specific species—while much evidence may be available for more descriptive purposes, such as for understanding species occurrence. However, for global-scale evidence syntheses with a broad scope, such as those conducted by the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services, incorporating non-English-language scientific knowledge should become the norm. Generating new scientific knowledge through individual studies requires sizable financial investment, as well as associated time costs [29]. Therefore, making better use of existing knowledge that has yet to be fully utilised due to the language of publication should be a cost- and time-efficient approach for filling gaps and rectifying biases in the evidence base for tackling urgent global challenges. In 1922 the philosopher Ludwig Wittgenstein stated “Die Grenzen meiner Sprache bedeuten die Grenzen meiner Welt” (the limits of my language mean the limits of my world) [30]. Hundred years on, his quote still seems applicable to science today. Scientific communities should stretch the limits of our shared knowledge, and its benefits, by uncovering knowledge that has long been accumulating and continues to be produced in languages other than English.
Materials and Methods
Searches for non-English-language studies on the effectiveness of conservation interventions
Objective of the searches
The searches aimed to identify peer-reviewed scientific studies (a study is defined as a paper published in a peer-reviewed journal) written in a language other than English that tested the effectiveness of one or more conservation interventions for any species group or habitat. Our search strategy was based on the protocol for discipline-wide literature searching, established and adopted for the development of the Conservation Evidence database [17] and published elsewhere [31]. Discipline-wide literature searching involves first identifying literature sources (peer-reviewed academic journals in our case) that are likely to contain relevant information, and manually scanning titles and abstracts (or summaries) of every document in those sources. We adopted discipline-wide literature searching, rather than systematic mapping/reviewing, as the former approach does not depend on search term choice, and can identify novel conservation interventions that would not necessarily have been identified on the basis of predetermined criteria for study inclusion [32]. For more details on the Conservation Evidence database, see Section English-language studies on the effectiveness of conservation interventions. Although non-English-language grey literature (e.g., reports, theses, etc.) could also play an important role in environmental evidence syntheses [33], our searches focused only on studies published in peer-reviewed journals, so as to enable a comparison between eligible non-English-language studies and peer-reviewed English-language studies stored in the Conservation Evidence database.
Selecting languages
We originally aimed to cover the top 15 non-English languages on the basis of the number of conservation-related publications, provided in Table S1 of [16]. However, we could not find native speakers of Swedish and Dutch who were willing to collaborate, and thus both languages were excluded from our searches. Instead, we were able to cover three additional languages (Arabic, Hungarian, and Ukrainian). In total, our searches covered 16 languages (Table S2).
Searchers
Our searches were conducted by a total of 38 native speakers of the 16 languages covered (hereafter referred to as searchers). The number of searchers for each language ranged from one to six (see Table S2 for more detail). We used a range of approaches (e.g., known networks, social media, e-mail lists, and the website of the translatE project: https://translatesciences.com/) to recruit our searchers. The searchers were required to be at least undertaking or have a bachelor’s degree, but often had higher research (i.e., master’s or doctorate) degrees, in a relevant discipline (e.g., ecology, biodiversity conservation, etc.), to ensure that they could fully understand the relevant studies and assess their eligibility during screening.
Before starting the searches, every searcher was trained through the following four steps. First, searchers were directed to read through a guidance document detailing the objectives and processes of the searches. Second, searchers were also requested to read and understand the full criteria for selecting eligible studies during the searches, which were described in detail, together with examples of 14 eligible and five non-eligible English-language studies, each with a full explanation on why it was or was not eligible. Third, searchers were advised to visit the Conservation Evidence website, particularly the page providing training resources (https://www.conservationevidence.com/content/page/89), and familiarise themselves with eligible English-language studies that tested the effectiveness of conservation interventions (listed at: https://www.conservationevidence.com/data/studies). Finally, all searchers were asked to conduct a test study screening, where they were requested to read the metadata (publication year, journal, volume, issue, authors, title, and abstract) of 51 English-language papers (29 from volume 200 (2016) of Biological Conservation, and 22 from volume 30 (2016) of Conservation Biology), which included a total of 14 eligible studies, decide if each study was eligible or not, and provide the full reasoning for their decisions. The outcome of the test screening was examined by either T.A., V.B.E, or other members of the Conservation Evidence project, who provided searchers with feedback.
Identifying and selecting journals for each language
We first identified and listed peer-reviewed academic journals published in each language which were likely to contain eligible studies. This process involved one to four researchers for each language (all native speakers of the target language, with at least a bachelor’s, and often higher, research degree; this often included the searchers) from relevant disciplines (see Table S2 for more detail), who used a range of approaches (e.g., personal knowledge, opinions from colleagues, local literature databases, web searches, etc.) to identify as many potentially relevant journals as possible. All journals identified were then grouped into three categories: “very relevant” (often journals in ecology and biodiversity conservation, as well as taxonomic journals, such as those in ornithology, mammalogy, herpetology, plant sciences, etc.), “relevant” (mostly journals in relevant disciplines, such as agricultural/forest sciences and general zoology), and “maybe relevant” (all others). Subsequent searches aimed to at least cover all journals categorised as “very relevant” and, when possible, those in the other two categories (see Data S1 for the list of all journals searched).
Screening papers in each journal
Searches for eligible studies in each journal were conducted by manually scanning the title and abstract (or summary) of every peer-reviewed non-English-language paper published in the journal, and by reading the main text of all papers for which the title and/or abstract were suggestive of fulfilling the eligibility criteria (fully described below). All papers that appeared to meet the eligibility criteria were identified as potentially eligible studies, with the relevant metadata recorded (see Data coding), and were then passed on to the validation process (see Study validity assessment). The journals were searched backwards from the latest volume, either to the earliest published volume or going back ten years for long-running journals (see Data S1 for publication years covered for each journal). We also recorded the total number of papers screened in each journal.
The following eligibility criteria, which were developed and published by the Conservation Evidence project (https://osf.io/mz5rx/), were used.
Criteria A: Include studies that measure the effect of an intervention that might be done to conserve biodiversity
Does this study measure the effect of an intervention that is or was under the control of humans, on wild taxa (including captives), habitats, or invasive/problem taxa? If yes, go to 3. If no, go to 2.
Does this study measure the effect of an intervention that is or was under the control of humans, on human behaviour that is relevant to conserving biodiversity? If yes, go to Criteria B. If no, the study will be excluded.
Could the intervention be put in place by a conservationist/decision maker to protect, manage, restore or reduce impacts of threats to wild taxa or habitats, or control or mitigate the impact of the invasive/problem taxon on wild taxa or habitats? If yes, the study will be included. If no, the study will be excluded.
Eligible populations or subjects
Included: Individuals, populations, species, or communities of wild taxa, habitats or invasive/problem taxa.
Excluded: Domestic/agricultural species.
Eligible interventions
Included: Interventions that are carried out by people and could be put in place for conservation. Interventions within the scope of the searches include, but not limited to:
Clear management interventions, e.g., closing a cave to tourism, prescribed burning, mowing, controlling invasive species, creating or restoring habitats,
International or national policies,
Reintroductions or management of wild species in captivity, and
Interventions that reduce human-wildlife conflict.
See Criteria B for interventions that have a measured outcome on human behaviour only. Also see https://www.conservationevidence.com/data/index for more examples of interventions.
Excluded:
Impacts of threats (interventions which remove threats would be included), impacts from natural processes (e.g., tree falls, natural fires), and impacts from background variation (e.g. soil type, vegetation, climate change).
Eligible outcomes
Included: Any outcome (can be negative, neutral or positive, does not have to be statistically significant) that is quantified and has implications for the health of individuals, populations, species, communities or habitats, including, but not limited to:
Individual health, condition or behaviour, including in captivity: e.g., growth, size, weight, stress, disease levels or immune function, movement, use of natural/artificial habitat/structure, range, predatory or nuisance behaviour that could lead to retaliatory action by humans.
Breeding: egg/offspring/seed/sperm production, sperm motility/viability after freezing, natural/artificial breeding success, birth rate, offspring condition/survival, and overall recruitment.
Genetics: genetic diversity, genetic suitability (e.g., adaptation to local conditions, use of flyways for migratory species etc.).
Life history: age/size at maturity, survival, mortality.
Population measures: number, abundance, density, presence/absence, biomass, movement, cover, age-structure, species distributions (only in response to a human action), disease prevalence, and sex ratio.
Community/habitat measures: species richness, diversity measures (including trait/functional diversity), community composition, community structure (e.g., trophic structure), area covered (e.g., by different habitat types), and physical habitat structure (e.g. rugosity, height, basal area).
Eligible types of study design
Included:
Studies with After, Before-After, Control-Impact, Before-After-Control-Impact, or Randomised Controlled Trial designs (using the definition provided in [34]). Literature reviews, systematic reviews, meta-analyses or short notes that review studies that fulfil the eligibility criteria are also included. Studies that use statistical/mechanistic/mathematical models to analyse real-world data or compare models to real-world situations are also included (if they otherwise fulfil the eligibility criteria).
Excluded:
Theoretical modelling studies, opinion pieces, correlations with habitat types where there is no test of a specific intervention by humans, or pure ecology (e.g., movement, distribution of species).
Criteria B: Include studies that measure the effect of an intervention that might be done to change human behaviour for the benefit of biodiversity
Does this study measure the effect of an intervention that is or was under human control on human behaviour (actual or intentional) which is likely to protect, manage, restore or reduce threats to wild taxa or habitats (including mitigating the impact of invasive/problem taxon on wild taxa or habitats)? If yes, go to 2. If no, the study will be excluded.
Could the intervention be put in place by a conservationist, manager or decision maker to change human behaviour? If yes, the study will be included. If no, the study will be excluded.
Eligible populations or subjects
Included: Actual or intentional human behaviour including self-reported behaviours. Change in human behaviour must be linked to outcomes for wild taxa or habitats.
Excluded: Human psychology (tolerance, knowledge, awareness, attitude, perceptions or beliefs). Changes in behaviour linked to outcomes for human benefit, even if these occurred under a conservation program (e.g., we would exclude a study demonstrating increased school attendance in villages under a community-based conservation program).
Eligible interventions
Included: Interventions that are under human control and change human behaviour, resulting in the conservation, management, and restoration of wild taxa or habitats. Interventions which are particularly likely to have a behaviour change outcome include, but are not limited to:
Enforcement: hunting restrictions, market inspections, increase number of rangers, patrols or frequency of patrols in, around or within protected areas, improve fencing/physical barriers, improve signage.
Behaviour change: promote alternative/sustainable livelihoods, payment for ecosystem services, ecotourism, poverty reduction, increased appreciation or knowledge, debunking misinformation, altering or re-enforcing local taboos, financial incentives.
Governance: protect or reward whistle-blowers, increase government transparency, ensure independence of judiciary, provide legal aid.
Market regulation: trade bans, taxation, supply chain transparency laws.
Consumer demand reduction: increase awareness or knowledge, fear appeals (negative association with undesirable product), benefit appeal (positive association with desirable behaviour), worldview framing, moral framing, employing decision defaults, providing decision support tools, simplifying advice to consumers, promoting desirable social norms, legislative prohibition.
Sustainable alternatives: certification schemes, artificial alternatives, sustainable alternatives.
New policies for conservation/protection.
Excluded: Impacts from climatic or other natural events. Studies with no intervention, e.g. correlating human personality traits with likelihood of conservation-related behaviours.
Eligible outcomes
Included: Any human behaviour outcome (can be negative, neutral or positive, does not have to be statistically significant) that is quantified and is likely to have an outcome on wild taxa or habitats, including, but not limited to:
Change in adverse behaviours (which directly threaten biodiversity), e.g., unsustainable hunting, burning, grazing, urban encroachment, creating noise, entering sensitive areas, polluting or dumping waste, clearing or habitat destruction, introducing invasive species.
Change in positive behaviours, e.g., uptake of alternative/sustainable livelihoods, number of households adopting sustainable practices, donations.
Change in policy or conservation methods, e.g., placement of protected areas, protection of key habitats/species.
Change in consumer or market behaviour, e.g., purchasing, consuming, buying, willingness to pay, selling, illegal trading, advertising, consumer fraud.
Behavioural intentions to do any of the above.
Eligible types of study design Same as Criteria A.
Data coding
From each of the studies that were identified by searchers as potentially eligible, the following metadata were extracted and recorded using a template file:
- Journal language
- Journal publication country
- Reference type (either original article, review, short note or others)
- Authors
- Publication year
- Title in English (if available) and in the non-English language
- Journal name in English (if available) and in the non-English language
- Volume / Issue / Pages
- Abstract in English (if available) and in the non-English language
- Keywords in English (if available) and in the non-English language
- Link to the article (URL, if available)
- Study site locations (coordinates; mean coordinates where a study had multiple sites, or city/state/province/country if coordinates were not available)
- Study design (either After, Before-After, Control-Impact, Before-After-Control-Impact, Randomised Controlled Trial, or review; using the definition of each design provided in [34])
- Broad species group(s) / habitat(s) studied
- Scientific name of study species (if available)
- Common name of study species in English and in the non-English language (if available)
- One-sentence summary in the form of: “This study tested the effect of [intervention(s)] on [measured outcome] of [target species or ecosystem(s)]” (e.g., “This study tested the effect of providing nest boxes on the breeding success of blue tits”)
The metadata were extracted largely by the searchers, but, for some languages where the searchers were not available, by other collaborators who are native speakers of the language and are at least undertaking or have a bachelor’s, but often higher research, degree in a relevant discipline (see Table S2 for more detail). They were all requested to first read and fully understand our guidance detailing the definitions of different study designs (provided in [34]) before starting data coding.
For all studies that were validated as eligible (see Study validity assessment), the recorded names of birds, mammals, and amphibians were standardised based on the lists of bird species names used by BirdLife International [35], and mammal and amphibian species names used by IUCN [36]. We focused on these three taxa for comparing study locations and species between languages because English-language studies testing the effectiveness of conservation interventions for these three taxa have extensively been searched using both discipline-wide literature searches and subject-wide evidence syntheses [17]. To identify species name synonyms we used the package ‘taxize’ [37] in R [38] with API keys generated at the NCBI (https://www.ncbi.nlm.nih.gov/account/) and IUCN (https://apiv3.iucnredlist.org/api/v3/token) websites.
Study validity assessment
The eligibility of each study that was identified as being potentially eligible was validated by at least one experienced literature searcher (assessors) at the Conservation Evidence project (see Table S2 for more detail), who regularly screen, identify, and summarise eligible studies using the same eligibility criteria (see Screening papers in each journal) but who mostly are not native speakers of each non-English language. This process was conducted by assessing the English-language title, abstract and one-sentence summary of each study identified by the searchers (see Data coding), and, where the validity could not be determined easily, also involved direct discussions between the relevant searchers and assessors to obtain clarification on the details of each study. Those studies that were deemed ineligible by the assessors were excluded from the final list of eligible studies in each language.
Limitations
Although, as described above, we adopted a search strategy that allowed us to identify eligible studies in as unbiased a way as possible, our search results can still suffer from some inevitable limitations:
· Language selection
Of the top 15 non-English languages on the basis of the number of conservation-related publications, our searches could not cover Swedish and Dutch. Nevertheless, we expect that the exclusion of these two languages only had a minimal effect on our conclusions, as conservation-related publications in these two languages were estimated to only constitute 0.87% of publications in the top 15 non-English languages [16], while we also covered three additional languages.
· Journal selection
Although we identified 465 journals in 16 languages, we were only able to screen 326 of them, as we prioritised journals ranked as “very relevant” and “relevant” for some languages when there was a shortage of searchers and/or their time that could be dedicated to the search process. Therefore, we assessed whether our choice of journals screened in each language was appropriate for identifying the most eligible studies in the language, by examining the “rank-abundance” curve for each language, where the x axis of the curve was the rank of searched journals according to the % of eligible studies (the journal with the highest % of eligible studies was given rank 1), and the y axis was the % of eligible studies. If a curve reached zero (i.e., there were almost no eligible studies) in lower-ranked journals, we interpreted it as an indication that sufficient coverage of journals had been reached for that language (see Fig. S1 for the result).
· Publication year selection
Searches for some long-running journals only went back ten years from the latest volume, thus potentially missing some eligible studies dating back further. We thus assessed the effects of excluding earlier volumes of long-running journals from our searches, by testing how the number of eligible studies changed over time in each journal (see Fig. 1 for the result).
· Possibility of missing eligible studies
We tried to identify as many eligible studies as possible in each language, by making sure that (i) every searcher was well qualified and trained before starting the searches (see Searchers) and (ii) when in doubt searchers keep, rather than reject, a study as potentially eligible, the validity of which was later assessed by independent experts (see Study validity assessment). Nevertheless, we cannot dismiss the possibility that some eligible studies were missed during the searches. This would have caused a potential underestimation of the number of eligible studies published in non-English languages. However, this should not undermine our main conclusion that scientific evidence published in non-English languages could fill gaps in the geographic and taxonomic coverage of English-language evidence for conservation.
· Potential variations in assessment outcomes of eligible studies and study designs among searchers
Although we did our best to train searchers to fully understand the eligibility criteria (see Screening papers in each journal) and the definition of different study designs (see Data coding), some inevitable variations may remain in the assessment outcomes of eligible studies and study designs among searchers. This would potentially affect the reported patterns in (i) the number and proportion of eligible studies among non-English languages (Fig. S1), and (ii) the proportion of different study designs among different languages (Fig. 2). Nevertheless, among-searcher variations in judgements should affect neither (i) yearly increases in the number of eligible non-English-language studies in each journal (Fig. 1), as the same journal was searched by a single searcher, nor (ii) the spatial and taxonomic complementarity between English- and non-English-language studies (Figs. 3 and 4), assuming that any such variations in assessment outcomes only affected a limited number of non-English-language studies, and thus have not drastically changed the overall patterns in the differences between English and non-English-language studies.
· Effects of publication bias
We focused only on studies published in peer-reviewed academic journals and thus did not consider the effects of publication bias, caused by ignoring grey literature, within each language, while recognizing that important scientific knowledge may also be published in non-English-language grey literature [33]. Therefore, it should be noted the conclusions of this study are limited to peer-reviewed studies published in academic journals.
English-language studies on the effectiveness of conservation interventions
To compare study characteristics (i.e., study design, study location and study species) between eligible English- and non-English-language studies, we used English-language studies stored in the Conservation Evidence database [17]. Those English-language studies were identified through a screening of peer-reviewed papers published in over 330 English-language academic journals including local and taxonomic journals (see the list at: https://www.conservationevidence.com/journalsearcher/english) based on the same eligibility criteria as described in the section “Screening papers in each journal” above (also see [17] for more detail). We extracted the metadata (including publication year, study site coordinates—mean coordinates where a study had multiple sites, study design, scientific and common names of study species) for each of the 4,412 English-language studies (Data S4) (including 284 studies on amphibians, 1,115 studies on birds, and 1,154 studies on mammals; Data S5) from the database on 11/12/2020. Again, here we defined a paper published in a peer-reviewed journal as a study. The Conservation Evidence database also stores some non-English-language peer-reviewed studies, most of which were identified incidentally by the project. Those non-English-language studies were also incorporated into our dataset of non-English-language studies, if they were in any of the 16 languages covered in this study (a total of 74 non-English-language studies, see records with “Source” being “Ad hoc” in Data S3). For birds, mammals, and amphibians, species names were standardised using the lists of bird, mammal, and amphibian species names used by BirdLife International [35] and the IUCN [36].
Analyses of eligible studies
Comparing the proportion of eligible studies in each journal
We first calculated the proportion of eligible studies for each non-English-language journal, by dividing the number of eligible studies by the total number of studies screened in the journal. Journals with 30 or fewer studies screened were excluded from this calculation, as the estimated proportions would be unreliable given the small sample size.
Testing yearly changes in the number of eligible non-English-language studies
To test whether the number of eligible non-English-language studies had changed over time, we focused only on journals with ten or more eligible studies, resulting in journals/studies in a total of 12 languages (shown in Fig. 1) being used in the following analysis. For each language, we fitted a generalised linear model (GLM) assuming a Poisson distribution with the number of eligible studies in each year in each journal as the response variable, and year and journal (for languages with more than one journal) as the explanatory variables. Journals were included in each model as a fixed, not random, effect, as the number of journals with ten or more eligible studies in each language was relatively small (nine for Japanese, five for German, and < 5 for all others), making it difficult to estimate the among-journal variance accurately in a mixed model [39].
Comparing study designs
To test whether there was a difference in study designs adopted between studies in different languages, we only included studies based on one of the following five designs: After, Before-After (BA), Control-Impact (CI), Before-After-Control-Impact (BACI), and Randomised Controlled Trial (RCT). These study designs were recorded as an ordinal variable with RCT being the least biased design, followed by BACI, CI, BA, and After, based on results from [24]. Considering that English-language studies in English-speaking countries (especially the UK and the US) may adopt more robust study designs than English-language studies in other countries, English-language studies were further divided into two groups; studies conducted in countries where English is an official language (“English – official”), and studies in all other countries (“English – others”), using information on countries’ official languages in [40]. We then fitted a cumulative link model using the ordinal package [41] in R, with ordered study designs in each study as the response variable and languages (16 non-English languages and “English – official”, compared to “English – others” as the reference category) and taxa (birds, mammals, and others, compared to amphibians as the reference category) as the explanatory variables.
Comparing study locations
To test whether there was a systematic bias in study locations between English- and non-English-language studies, we first calculated the number of studies for each language in each 2° × 2° grid cell. Studies without study coordinates were excluded from this calculation, leading to 4,254 English-language studies (including 267 studies on amphibians, 1,084 studies on birds, and 1,062 studies on mammals) and 1,202 non-English-language studies (including 53 studies on amphibians, 244 studies on birds, and 153 studies on mammals) being used in the following analysis. As the latest English-language studies on birds, amphibians, and mammals stored in the Conservation Evidence database were those published in 2011, 2012, and 2018, respectively, non-English-language studies after those years were excluded from the comparison of studies on each taxon, leading to 31 studies on amphibians, 182 studies on birds, and 146 studies on mammals being used in the analysis. We used a conditional autoregressive (CAR) model assuming a Poisson distribution to test the association between the number of non-English-language studies (the response variable) and the number of English-language studies (the explanatory variable) within each grid cell while accounting for spatial autocorrelation in residuals (see Data availability for the availability of the R code). We fitted the model to the data with the Markov chain Monte Carlo (MCMC) method in OpenBUGS 3.2.3 [42] and the R2OpenBUGS package [43] in R. We set prior distributions of parameters as non-informatively as possible, so as to produce estimates similar to those generated by a maximum likelihood method; we used an improper uniform distribution (i.e., a uniform distribution on an infinite interval) for the intercept following [44], a normal distribution with a mean of 0 and variance of 100 for the coefficient of the explanatory variable, and Gamma distributions with a mean of 1 and variance of 100 for the inverse of variance in an intrinsic Gaussian CAR distribution. We ran each MCMC algorithm with three chains with different initial values for 35,000 iterations with the first 5,000 discarded as burn-in and the reminder thinned to one in every 12 iterations to save storage space. Model convergence was checked with R-hat values.
Comparing species
To test whether there was a systematic bias in study species between English- and non-English-language studies, we first calculated the number of English- and non-English-language studies available for each species. We used generalised linear mixed models (GLMMs) assuming a Poisson distribution to test the association between the number of non-English-language studies (the response variable) and the number of English-language studies (the explanatory variable) for each species while accounting for phylogenetic autocorrelation by incorporating the family of each species as a random factor (see Data availability for the availability of the R code). The GLMMs were implemented in R with the lme4 package [45].
Other R packages used in the analyses and data visualization were: data.table [46], dplyr [47], gridExtra [48], mapdata [49], mcmcplots [50], MCMCvis [51], plyr [52], RColorBrewer [53], rgdal [54], readxl [55], tidyverse [56], viridis [57], and writexl [58].
Data availability
All data and code used in the analysis are available at: http://doi.org/10.17605/OSF.IO/Y94ZT.
Author contributions
Conceptualization: T.A., Formal analysis: T.A., V.B.E. and A.P.C., Funding acquisition: T.A., A.P.C., A. Báldi, S.B., K.J., J.K.-B, I.K., M.-C.L., V.M., J.P.N.G., P.J.N., A.N.R.V., M.-M.R., F.V., J.O.V., R.D.Z. and W.J.S., Investigation: T.A., V.B.E., A.P.C., M.A., A. Báldi, S.B., C.M., C.C.Y., M.B.D.K., L.G.O., P.F., M.G., N.H.A., K.J., J.K.-B., M.Ç.K.A., I.K., K. Kito, K. Konno, D.-L.L., Yang L., Yifan L., M.-C.L., V.M., J.P.N.G., P.J.N., E.N., J.M., R.R.Y., A.C.P.-B., I.L.P., D.L.R., A.L.R.S., A.N.R.V., R.R., M.-M.R., R.S., D.S., Ç.H. ., H.S., Y.S., Ş S.D.T., M.S.T., F.V., J.O.V., S.V., P.W., V.Z.G., R.D.Z. and W.Z., Methodology: T.A., V.B.E., A.P.C. and W.J.S., Project administration: T.A., V.B.E., K.W., E.T. and W.J.S., Supervision: T.A. and W.J.S., Validation: T.A., V.B.E., K.W., A. Berthinussen, A.J.B., N.A., P.A., W.H.M., N.O., S.O.P., R.R., K.A., G.S., R.K.S. and E.T., Visualization: T.A. and A.P.C., Writing – original draft: T.A., Writing – review & editing: T.A., V.B.E., A.P.C., K.W., M.A., A. Báldi, A. Berthinussen, S.B., A.J.B., C.M., C.C.Y., M.B.D.K., L.G.O., P.F., M.G., N.H.A., K.J., J.K.-B., M.Ç.K.A., I.K., K. Kito, K. Konno, D.-L.L., N.A., Yang L., Yifan L., M.-C.L., V.M., P.A., W.H.M., J.P.N.G., P.J.N., E.N., J.M., N.O., R.R.Y., S.O.P., A.C.P.-B., I.L.P., D.L.R., A.L.R.S., A.N.R.V., R.R., M.-M.R., K.A., R.S., D.S., Ç.H.Ş., H.S., G.S., Y.S., R.K.S., S.D.T., M.S.T., E.T., F.V., J.O.V., S.V., P.W., V.Z.G., R.D.Z., W.Z. and W.J.S.
Competing interests
The authors declare no competing interests.
Supporting information
Supplementary Text
In this study we screened 419,680 peer-reviewed papers in 326 journals, published in 16 languages, to identify non-English-language studies testing the effectiveness of interventions in biodiversity conservation. This enabled us to test four commonly held perceptions that: (i) the amount of relevant scientific evidence that is available only in non-English languages is negligible; (ii) the number of relevant studies being published in non-English languages has been decreasing over time; (iii) non-English-language studies are often based on less robust study designs than English-language studies; (iv) there is no bias in the scientific evidence provided between English-language studies and non-English-language studies, in terms of geographical and taxonomic coverage. Our findings provide novel, quantitative insights into how non-English-language scientific knowledge can contribute to environmental evidence syntheses and the implementation of much-needed evidence-based conservation globally.
The potential importance of non-English-language studies in evidence synthesis has long been explored in healthcare, where studies have reported systematic biases in statistical results between English- and non-English-language studies [59, 60], tested differences in study quality between languages [7], and assessed the effects of excluding non-English-language studies on the outcomes of meta-analyses [10, 11]. These attempts were focused almost exclusively on how including non-English-language studies might change the statistical results of meta-analyses, and have neither investigated temporal changes in relevant non-English-language evidence nor compared the characteristics of evidence provided between languages. However, non-English-language studies are also expected to provide scientific information that is not available in English-language studies, and thus expand the coverage of evidence that can be incorporated, especially in disciplines dealing with more geographically and taxonomically diverse targets and phenomena than healthcare [12].
Earlier studies have also attempted to quantify the importance of non-English-language scientific knowledge in ecological evidence syntheses. For example, 67% of the scientific literature identified in a systematic review on Japanese bats including many endemic and threatened species was in Japanese [61], literature searches in Spanish increased the amount of scientific literature on interactions between birds and wind farms by 11% [62], and 65% of the literature included in a systematic review on China’s Belt and Road Initiative, a continental-scale infrastructure development that potentially has disastrous consequences for biodiversity in the region, was in Chinese [63]. Despite the importance of their findings, most studies are limited only to a specific research topic and a single non-English language, restricting the generalizability of their findings.
A few studies have also investigated the availability of ecological knowledge published in multiple non-English languages. Amano et al. [16] showed that up to 35% of scientific documents on biodiversity conservation published in 2014 were in 15 non-English languages; however the study did not investigate and compare the detail of those scientific documents published in different languages. Another example is the study conducted by Angulo et al [64], which compiled data on the global economic costs of invasive alien species reported in 15 non-English languages, and showed that non-English-language sources (i) capture a greater amount of data than English-language sources, (ii) fill in geographic and taxonomic gaps in English-language sources, and (iii) increase the global cost estimate of invasions by 16.6%. This study, while providing important insights into the role of non-English-language sources in developing a specific database, lacks assessments of (i) temporal changes in non-English-language sources, and (ii) differences in study quality between languages. These limitations are critical in rigorously testing the role of non-English-language knowledge in evidence synthesis, as the importance of non-English-language knowledge could wrongly be overstated if (i) the amount of newly published non-English-language knowledge is decreasing (as is argued in [19]), or (ii) the quality of non-English-language knowledge is lower than that of English-language knowledge (as is shown in several healthcare studies [7]).
In contrast, our study revealed that the number of relevant non-English-language studies is increasing in many languages, showing that non-English-language science will continue playing a crucial role, while there seems to be a trade-off between evidence quality and availability in regions and species with little English-language evidence, highlighting a future research priority for making the best use of evidence with varying qualities and degrees of relevance to a specific context.
Supplementary Data (available at: http://doi.org/10.17605/OSF.IO/Y94ZT)
Data S1
The list of non-English-language peer-reviewed journals related to biodiversity conservation identified in this study. The explanations of column names are as follows: Language: journal publication language, Country: journal publication country, Journal title in English: journal title in English, Journal title in non-English language: journal title in the non-English language, First publication year: the first publication year, Latest publication year: the latest publication year (as of March 2021), Link (URL): link to the journal website, Research areas/taxa: broad research area and taxa covered in the journal, Searcher: searcher name, Years screened first: the publication year of the first volume screened, Years screened last: the publication year of the last volume screened, Years screened total: the number of years screened, Volumes screened: the number of volumes screened, Number of papers screened: the number of studies screened, Number of papers id as relevant by collaborators: the number of studies initially identified as eligible by searchers, Number of papers validated as relevant: the number of studies validated as eligible, Number of papers added ad hoc from CE dataset: the number of studies added from the Conservation Evidence database, Total relevant: the total number of eligible studies, Comments: any other relevant notes.
Data S2
The list of 1,234 non-English-language studies identified as eligible in this study. The explanations of column names are as follows: Paper ID: study ID, Translator Name: searcher name, Language: study publication language, Journal Country: journal publication country, Reference Type: the type of publications (e.g., journal article, review, etc), Authors (separate with//): the name of authors, Year: publication year, Title – English: title in English, Title - non-English language: title in the non-English language, Journal: journal name, Volume: volume, Issue: issue, Pages: pages, Abstract – English: abstract in English, Abstract - non-English: abstract in the non-English language, Keywords – English: keywords in English, Keywords - non-English: keywords in the non-English language, Broad species group(s)/ habitat(s)/ ecosystem service(s): broad species group(s) / habitat(s) studied, Species Scientific Name: scientific name of study species, Species English Name: common name of study species in English, Species No-English Name: common name of study species in the non-English language, Study design: study design adopted, Mean Lat: mean latitude of the study site(s), Mean Long: mean longitude of the study site(s), City/state or province/country: city/state/province/country of the study site(s), DOI: Digital Object Identifier, Link (URL): link to the paper.
Data S3
The list of species studied in the 1,234 non-English-language eligible studies. The explanations of column names are as follows: Paper ID: study ID, Language: study publication language, IUCN: scientific name of study species used by the International Union for Conservation of Nature (IUCN name), Species Scientific Name: scientific name of study species recorded by searchers, Common name: common name of study species identified with the package ‘taxize’ in R, Species English Name: common name of study species in English recorded by searchers, Species Non-English Name: common name of study species in the non-English language recorded by searchers, Taxa: taxonomic group identified based on the IUCN name, Broad species group(s)/ habitat(s)/ ecosystem service(s): broad species group(s) / habitat(s) studied, Study design: study design adopted, Mean Lat: mean latitude of the study site(s), Mean Long: mean longitude of the study site(s), City/state or province/country: city/state/province/country of the study site(s), Journal: journal name, Journal Country: journal publication country, Source: the method of identifying the study (systematic review: discipline-wide literature searching, Ad hoc: identified in the Conservation Evidence database) Year: publication year.
Data S4
The list of species studied in the 4,412 English-language studies stored in the Conservation Evidence database. The explanations of column names are as follows: rowed: record ID, pageid: study ID, journal_match_scimago: journal name used in the Scimago Journal Rank, journal: journal name, syn: Conservation Evidence synopsis including the study, int: conservation intervention tested, before: if the study has a Before element, controlled: if the study has control(s), randomized: if replications are randomized, review: if the study is based on a review or not, pubdate: publication year, lat: latitude of the study site(s), long: longitude of the study site(s), country: country of the study site(s), species: specific name of the study species, genus: generic name of the study species, family: family of the study species, order: order of the study species, class: class of the study species, binom: scientific name of the study species, pubtype: study publication type, original_title: paper title, ref_startpage: start page of the paper, ref_endpage: end page of the paper, ref_vol: volume published, ref_issue: issue published, ref_doi: Digital Object Identifier, ref_citation: paper citation, ref_authorstring: authors.
Data S5
The list of amphibian, bird and mammal species studied in the English-language studies stored in the Conservation Evidence database. The explanations of column names are as follows: pageid: study ID, syn: Conservation Evidence synopsis including the study, int: conservation intervention tested, before: if the study has a Before element, controlled: if the study has control(s), randomized: if replications are randomized, review: if the study is based on a review or not, pubdate: publication year, lat: latitude of the study site(s), long: longitude of the study site(s), country: country of the study site(s), species: specific name of the study species, genus: generic name of the study species, family: family of the study species, order: order of the study species, class: class of the study species, binom: scientific name of the study species (standardised based on the names used by the International Union for Conservation of Nature), habitat: broad habitat type studied, authors: authors, journal: journal name.
Acknowledgements
Thanks to I. Mangold and H. Korn for their help during the data collection, and M. Amano for all the support. This communication reflects only the authors’ view and any of the funders is not responsible for any use that may be made of the information it contains.