Abstract
Protected areas currently cover about 15% of the global land area and constitute one of the main tools in biodiversity conservation. Quantifying their effectiveness at protecting species from decline and local extinction involves comparing protected with counterfactual unprotected sites representing “what would have happened to protected sites if they had not been protected”. Most studies are based on pairwise comparisons, using as counterfactuals neighbour sites to protected areas, but this choice is often subjective and may be prone to biases. An alternative is to use large-scale biodiversity monitoring datasets, which whereby the effect of protected areas is analysed statistically by controlling for landscape differences between protected and unprotected sites, allowing a more targeted and clearly defined measure of protected areas effect. Here we use the North American Breeding Bird Survey dataset as a case study to investigate protected areas effectiveness in conserving bird assemblages. We analysed the effect of protected areas on species’ richness, assemblage-level abundance and abundance of individual species by modelling how these metrics relate to the proportion of each site that is protected, while controlling for site habitat, altitude, productivity and spatial autocorrelation. At the assemblage level, we found no relationship between protection and species richness or overall abundance. At the species level, we found that species that avoid human activities tend to be favoured by protected areas are the one avoiding human activities. Moreover, we found that forest protected areas presented higher abundances of forest species, making the assemblage more typical of this habitat. We did not find that declining species were particularly favoured by protected areas. Our results highlight the complexity of answering the question of protected areas effectiveness, and the necessity to define clearly metrics measured and the controls used.
Introduction
The increasing human footprint on natural ecosystems is leading to major declines in species’ populations (McRae et al., 2016) and has already resulted in thousands of extinctions (IUCN, 2018), to such an extent that Ceballos et al. (2017) characterised current times as a period of “biodiversity annihilation”. Habitat loss and degradation are the most important pressures on biodiversity (Vié et al., 2009; Balmford and Bond, 2005), as a result of anthropogenic activities such as agriculture, urbanisation, industry, transport and recreation (Foley et al., 2005). The most evident response to these threats is to establish areas with restricted, or even no human activities, i.e., to create protected areas (PAs). Modern PAs have their origins in the 19th century and currently represent the most important conservation tool, with about 15% of the global land area already protected to some extent, and coverage planned to reach 17% by 2020 (UNEP-WCMC IUCN, 2016).
Understanding the extent to which PAs are being effective as biodiversity conservation tools is fundamental for guiding future conservation efforts. Accordingly, there is a substantial and large literature on PA effectiveness: as of the 1st October 2018, 260 publications in the Web of Science included in their title “protected AND area* AND effective*”. However, within this literature there are disparate approaches to the concept of “effectiveness”.
A first set of studies questions whether PAs are effective at representing species or ecosystems, using gap analyses for measuring the overlap between PAs and the distributions of species or ecosystem types (e.g., Rodrigues et al., 2004; Brooks et al., 2004). These studies do not directly quantify the effectiveness of PAs at conserving biodiversity, but the extent to which species or ecosystems are buffered from human impacts under the assumption that PAs are highly effective in doing so. A second set of studies focuses on the means employed locally by PA managers in order to protect biodiversity, for example in terms of staff or money (e.g., Leverington et al., 2010). These analyses do not directly measure PA effectiveness in reducing human impacts, but rather the resources allocated to this purpose. A third type of studies quantifies the effectiveness of PAs at preventing the conversion of natural ecosystems, typically by comparing land use change (e.g., deforestation rates) in protected versus unprotected areas (Nelson and Chomitz, 2009; Andam et al., 2008). These studies quantify PA effects at the habitat or ecosystem level, rather than at the species level. Finally, a set of analyses focuses on measuring the effect of PAs on species themselves, either on the diversity of assemblages or on the abundance of individual species, typically by contrasting protected versus unprotected sites (e.g. Coetzee et al., 2014; Gray et al., 2016; Devictor et al., 2007, discussed below). This fourth approach to PA effectiveness is the focus of the present study.
Assessing the effectiveness of PAs in conserving species can be implemented by comparing population trends (e.g. Gamero et al., 2017; Devictor et al., 2007; Pellissier et al., 2013). Indeed, if PAs are effective, populations in them are expected to be better buffered from threats and thus decline less, or even to increase more, than those outside. Trends however can be misleading, because they are calculated in relation to a reference date (that seldom precedes all anthropogenic impacts) and because they are measured as percentages (which emphasise changes in small numbers). Hence, for example, given a species in three sites: one where it remained stable at 1000 individuals; a second where it initially declined from 1000 to 10 and recently increased to 20; and a third where it first declined from 1000 to 800 and recently recovered to 1000. In terms of recent trends, the second site appears by far the most effective, even though it has the most depleted population, and even though in absolute numbers the population increase in the third site is 20 times more important. In this study, we focus instead on measures of PA effectiveness that assess current state, namely by contrasting population abundances and species diversity (e.g. Coetzee et al., 2014; Kerbiriou et al., 2018; Devictor et al., 2007). Indeed, if PAs have been effective in conserving species, we expect that over time that translates into higher absolute population abundances than in counterfactual areas, as well (if local extinctions have been prevented) in species diversity.
Three recent studies investigated the effects of PAs on the state of species abundance and/or diversity, through meta-analyses of studies that made pairwise comparisons between protected and unprotected sites (Geldmann et al., 2013; Coetzee et al., 2014; Gray et al., 2016). The underlying studies used in these meta-analyses did not necessarily aim to measure PA effectiveness, more often they investigated the effects of anthropogenic pressure, using PAs as benchmarks (e.g. Sinclair et al., 2002; Bihn et al., 2008; Wunderle et al., 2006, all used in Coetzee meta-analysis). The meta-analyses considered that unprotected sites acts as counterfactuals to the protected sites (i.e., by assuming that the latter would be in a similar condition to the former had it not been protected), measuring the effect of protection as the observed difference between the two. Pairwise comparisons often compare neighbouring sites, which presents the advantage of ensuring that both have broadly similar environmental characteristics (e.g. same climate), but do not necessarily take in account the fact that PAs tend to be biased in their location towards higher altitudes and lower productivity areas (Joppa and Pfaff, 2009). To account for this, Gray et al. (2016) controlled for the differences in altitude, slope and agricultural suitability. Controlling for these factors means that their results are less influenced by PAs’ location biases and, therefore, that they reflect more strongly the effects of protection itself. Another potential bias resulting from pairwise comparisons of neighbouring sites arises from the leakage effect, whereby the human activities that would have taken place inside a PA are displaced to areas around it, artificially inflating the perceived effectiveness of PAs (Ewers and Rodrigues, 2008). This effect is difficult to control for, but should be reduced if the counterfactual sites are not immediately adjacent to the PAs.
An important decision when choosing a suitable spatial counterfactual to a PA, one that strongly affects the definition and thus the measure of PA effectiveness, is whether to control for habitat type or not. Indeed, not considering it could lead to comparing sites that are not expected to have similar biodiversity regardless of protection (e.g. protected grassland vs unprotected forest), while not considering it overlooks the effect of PAs through preventing habitat changes (e.g. deforestation or urbanization). For instance, given a hypothetical PA covering a natural grassland, possible counterfactuals include an unprotected natural grassland (same habitat, but unprotected), as well as a diversity of unprotected sites with different habitats, for example an extensive pasture (same vegetation structure, but with relatively low-level anthropogenic use), an herbaceous cropland (same vegetation structure but highly transformed), or an urbanised area (a wholly different ecosystem). This choice is certain to have a major impact on the differences observed, and thus on the measure of PA effectiveness, but it is not necessarily obvious what the best counterfactual should be. In theory, it is the site that best represents “what would have happened to the PA in the absence of protection”; in practice, this is not necessarily easily determined. All three meta-analyses include comparisons where habitat has not been controlled for, meaning that the counterfactual’s habitat may be different or similar to the protected site’s habitat. Additionally, a subset of Gray et al. (2016)’s analyses focuses on comparisons between protected and unprotected sites with matched habitats. In this, the measure of PA effectiveness concerns protection from habitat degradation rather than protection from habitat conversion.
Another key consideration in analysing PA effectiveness is the biodiversity metrics of interest. The three meta-analyses applied a diversity of metrics, some at the level of species’ assemblages, some focused on individual species. Gray et al. (2016) used only assemblage-level metrics and found higher species richness and overall abundance inside PAs than outside, but no difference in rarefaction-based richness (i.e. number of species for a given number of individuals) nor in the proportion of endemic species. When matching sites with similar habitats, species richness was only higher in young and small PAs than in unprotected sites (no difference between other protected and unprotected sites), suggesting that the effect of PAs on habitat degradation was light. Conversely, Geldmann et al. (2013) considered only species-level metrics (presence, abundance, nest survival) and found contrasted but mainly positive effects of PAs. Finally, Coetzee et al. (2014) considered both levels; at the assemblage level, they found higher species richness and overall abundance in protected than in unprotected sites; at the species level, they found that individual species abundances were typically higher inside PAs.
In this study, we use a different approach for quantifying PA effectiveness, one which is not based on pairwise comparisons, but instead takes advantage of a large dataset compiling bird counts across a near-continental area: the North American Breeding Bird Survey (Pardieck et al., 2017). This approach has already been used in other geographical areas, with other datasets (e.g. Devictor et al., 2007 with French birds; Kerbiriou et al., 2018 with French bats; Duckworth and Altwegg, 2018 with South-African birds) with heterogeneous results but mainly showing positive effects of PAs. In this approach, instead of pairing sites, the effect of PAs is quantified through statistical models in which covariates control for differences between protected and unprotected sites. This removes the subjectivity in the choice of counterfactuals, by making it clear which variables are controlled for, and the measure of effectiveness being investigated. In our study, we control for altitude and productivity in order to reduce the effect of PA location biases. We estimate PA effectiveness on two levels of biodiversity: on species’ assemblages, through indices of richness and summed abundance; and on individual species, by estimating the effect of PAs on species’ abundance for the most common species. At the assemblage level, we expect to find higher species diversity inside PAs. Indeed, as human activities are causing species population declines and local extinctions (Ceballos et al., 2017), and as PAs are expected to buffer against these activities, this should predictably lead to overall higher species richness and higher total abundance inside PAs, as found by Coetzee et al. (2014) and Gray et al. (2016). At the species level, we expect individual species’ abundances to be higher in PAs.
However, given differences in species’ habitat requirements, this result cannot be expected to hold universally (i.e., species are not all expected to be more abundant in all PAs). For example, we expect protected forests to have a positive effect on forest species, but not on grassland species. To take this into account, we control in our analyses for broad vegetation structure (forest, shrub, herbaceous), by investigating separately the effects of PAs dominated by a particular vegetation structure on species with different habitat requirements. Additionally, we expect species with overall declining populations (thus more affected by anthropogenic activities), and species that avoid human presence (more sensitive to human disturbance) to present higher abundances inside PAs.
Methods
As stated in the introduction, in this study we will use the term PAs effectiveness as the difference in diversity or abundance between protected and unprotected sites, acknowledging that it includes both effectiveness to select the most interesting sites for conservation when implementing PAs and the effectiveness in create more positive or less negative biodiversity trends inside PAs.
Bird data
We used data from the North-American Breeding Bird Survey (BBS), a long-term volunteer-based monitoring scheme in Canada, the USA, and Mexico (Pardieck et al., 2017). Here we studied only Canada and the USA, as few Mexican routes are monitored. This program is based on the annual monitoring of 25-mile routes during the breeding season. Each route is split into 50 stops; at each stop, the observer counts every bird heard or seen during three minutes, before moving to the next stop.
Given the length of BBS routes, they often intersect multiple land use types (e.g. forested, urban, agriculture, with different bird assemblages), and they are rarely wholly contained within protected sites (most of the routes that cross PAs do so only in small fractions of their length). As a result, whole BBS routes are not particularly suited sampling units for investigating how PAs affect bird species. We chose instead to focus on small sections of BBS routes – sequences of five stops, covering about 2.5 miles – in order to obtain field sampling units that are less heterogeneous in land type and for which there is a stronger correspondence between the presence or PAs and the bird assemblages detected. For each route, we only used the first sequence of five stops, because the only precisely georeferenced point we had access to was the starting stop of each route. Indeed, even if in principle additional stops are spaced about 0.5 miles from each other, in practice this distance can vary, making the location of additional stops in each route progressively more imprecise. Henceforth, and for simplicity, we use the term “routes” to refer to these initial sections of five stops rather than to entire routes.
We excluded bird taxa that are not well detected by this diurnal road-based monitoring scheme (aquatic and nocturnal birds), those that correspond to non-indigenous species, and hybrids. Overall, we analysed 400 species.
We focused on routes sampled at least 5 years between 2007 and 2016, obtaining a set of 3,427 routes analysed. For routes sampled more than five years, we analysed only five (randomly selected) years of data, thus ensuring a consistent sampling effort across all routes. For each species, the abundances were summed across the five points and the five years, giving a single value per species per route. We winsorized the abundances of each species (i.e., values above 95% quantiles were reduced to the 95% quantile value) to limit the impact of extreme values.
Landscape data
For each route, we analysed the properties of the landscape within a 500 m buffer around the route’s 2.5-mile track (total area ca. 6 km2), which we considered as a suitable description of the environment affecting the composition of birds detected by the BBS and which corresponds broadly to the bird detection radius of the BBS. Small et al. (2012) showed that the immediate landscape composition (buffer of 0.4 km) of BBS routes was similar to large-scale landscape composition (buffer of 10 km), so this choice is not expected to strongly affect the results.
Protected area is defined by the IUCN as “a clearly defined geographical space, recognised, dedicated and managed, through legal or other effective means, to achieve the long term conservation of nature with associated ecosystem services and cultural values”. They are categorised by the IUCN within seven categories based on their protection level from Ia “strictly protected areas set aside to protect biodiversity […], where human visitation, use and impacts are strictly controlled and limited to ensure protection of the conservation values.” to VI which “conserve ecosystems and habitats together with associated cultural values and traditional natural resource management systems. They are generally large, with most of the area in a natural condition, where a proportion is under sustainable natural resource management and where low-level non-industrial use of natural resources compatible with nature conservation is seen as one of the main aims of the area” (UNEP-WCMC and IUCN, 2018). We used the PAs’ shapefile, including both locations and IUCN categories of PAs, which was provided by the World Database on Protected Areas (UNEP-WCMC and IUCN, 2018). We calculated the proportion of area inside each route’s buffer that falls within a PA (all IUCN-categories combined, and dissolved to avoid double-counting of areas under multiple PA designations). We have also run analyses considering stricter PAs only (categories I-IV), as the effectiveness can vary with protection level.
For each route, we obtained values according to four environmental variables: net primary productivity, altitude, human footprint, and type of vegetation structure. The first three are continuous variables, available as raster files, and we obtained a mean values across all pixels that overlap the respective buffer: net primary productivity as the mean during spring months (Mars to June) between 2004 and 2015 according to the monthly Net Primary Productivity Terra/Modis (NASA, 2017; resolution 0.1 degree, about 62 km2 at 45°N); altitude using GLOBE Digital Elevation Model (National Geophysical Data Center, 1999; resolution 0.008 degree, about 0.40 km2 at 45°N); human footprint from the 2009 Global terrestrial Human Footprint (Venter et al., 2016; resolution 0.01*0.008, about 0.50 km2 at 45°N). We defined the vegetation structure as a categorical variable with three types: forest, shrub and herbaceous. We started by reclassifying the land cover classes in the Global Land Cover 2000 layer (Bartholomé and Belward, 2005; resolution 0.009 degree, about 0.50 km2 at 45°N) into the three vegetation structure types: forest from land cover classes 1-9 (N=1,749 routes); shrub, 11-12 (N=409); herbaceous, 13-16 (which includes croplands; N=1,140). We then obtained the main vegetation structure type for each route as the dominant in the buffer. Routes which were dominated by other land use classes (burned trees, 10; mosaic, 17-18; bare areas, 19; water areas 20-21; artificial, 22) were not analysed because too scarce. Routes used in analyses are mapped in Appendix S1.
Statistical analyses
We estimated the effect of PAs on each of two assemblage indices (species richness and summed abundance) and on the abundance of individual species using General Additive Models (GAMs). Models all had identical structures, with the response variable modelled as function of the proportion of PAs inside the buffer, interacting with vegetation structure type. We added smoothed terms controlling for productivity and altitude, as PAs are globally biased towards high altitude and low productivity areas (Joppa and Pfaff, 2009), as well as longitude and latitude in order to correct for spatial autocorrelation:
Response ~ PA * vegetation + s(productivity, altitude, longitude, latitude)
Assemblage level
For each route, and across all 400 bird species analysed, we calculated two assemblage indices, in each case using the cumulative number of species or individuals seen across the 5 stops, over 5 years: species richness (μ = 28.6 ± 9.5 species); summed bird abundance across all species (μ = 249 ± 88 individuals). We then used a GAM to model each of these two assemblage variables against the above-mentioned covariates, assuming a Gaussian distribution for richness and a negative binomial distribution for abundance.
Species level
We excluded the rarest species from this analysis, keeping only the 149 species observed on more than 100 routes, in order to have enough statistical power. For each species, we only analysed routes within the species’ distribution within our study area. We obtained an approximation of this distribution by delimiting the 90 % spatial kernel of the routes where the species was observed, using the ‘adehabitat’ R package (Calenge, 2006). We treated all routes inside the kernel where the species was not observed as having zero abundance.
We modelled each species’ abundance using a GAM as mentioned above, with a Poisson distribution. We then calculated for each species a “PA effect” (PAE), measured as the difference in predicted abundance between a fully protected and an unprotected route with all control variables fixed to their median values. We calculated PAE separately for each of the three types of vegetation structure, to obtain for each species a value of PAEFor for routes dominated by forest, PAEShrub for shrub routes, and PAEHerb for herbaceous routes.
For each type of vegetation structure, we studied PAE values in order to understand the factors explaining which species are favoured or not by PAs. To do so, we used a linear model (LM) and two phylogenetic linear models (phyBM and phyL, see below) with species-level covariates. We considered three covariates: species’ habitat preference, population trend, and human-affinity. We extracted from Del Hoyo et al. (2013) species’ main habitat (11 categories; see Fig.2). We used species’ population trends in North America between 1966 and 2015, calculated for each species by Sauer et al. (2017) from the BBS data (negative number for declining species, positive for increasing species). We winsorized these values, folding down the 2.5% extreme values on each side, bringing estimates to a Gaussian distribution. Finally, we estimated for each species a human-affinity index, as the median human footprint of the routes where the species was observed, weighted by species’ abundance on the route.
The two phylogenetic models used are the Brownian motion model (phyBM) and the Lambda model (phyL), both implemented in the ‘phylolm’ R package (Tung Ho and Ané, 2014). To obtain the bird phylogeny, we selected randomly 100 phylogenetic trees over 10,000 from Jetz et al. (2012) and calculated a maximum clade credibility tree using Tree Annotator from Mr Bayes (Drummond et al., 2012) with no burnin, and node heights calculated with the median.
Results
Assemblage-level analyses
At the assemblage level, species richness and summed abundance differed very significantly between vegetation structure types (respectively P<2.10−16, P=1.10−6), underlying the importance of accounting for habitat differences when studying PAs effect.
However, neither species richness, nor summed abundance were significantly affected by the proportion of PAs in the buffer (respectively P=0.13, P=0.13), or its interaction with vegetation structure type (respectively P=0.20, P=0.29). This lack of significance between assemblage indices and the proportion of PAs in the buffer was also true when not controlling for vegetation structure (respectively P=0.68, P=0.058).
Species-level analyses
PAEFor – the predicted difference in a given species’ abundance between protected versus unprotected forest routes – differed significantly depending on the species’ main habitat, under both LM and phyL models but not under phyBM model. Hence, the first two models indicate that within forest routes, species with any type of forest as main habitat (mixed, deciduous, forest, conifer; Table 1 and Fig.2) are predicted to have significantly higher abundances when routes are protected. We found no significant PA effect within forest routes for species favouring other habitat types.
In all models, species’ population trends between 1966 and 2015 did not significantly explain PAEFor (Table 1). In contrast, species’ human-affinity was significantly negatively correlated with PAEFor (i.e., species with lower affinity to humans had higher effects of PAs in forested routes; Table 1, Fig.3). This effect was also significant when only forest species were considered (green dots in Fig.3; see Supporting Information in Appendix S2 for additional test).
The effect of PAs within shrub routes (PAEShrub) and within herbaceous routes (PAEHerb) was not affected by species’ main habitat under any of the models (Supporting information, Appendix S3). PAEShrub decreased significantly with species’ trend under all models (i.e., declining species had higher effects of PAs in shrub routes), whereas it decreased with human-affinity only under model phyBM. PAEHerb was not significantly correlated with any of the three covariates.
These results, however, need to be interpreted taking into account that shrub or herbaceous protected routes were rare in our dataset: on average, each species’ kernel included only 10 shrub and 7 herbaceous routes protected by 50% or more, contrasted with 60 protected forest routes (Fig.1; see Appendix S4 in Supporting Information). The lack of significance in models with PAEShrub and PAEHerb might thus be due to the limited number of protected routes in the sample, whereas the significant correlations between PAEShrub and both species’ trends and human-affinity might not be robust.
Results, both at the assemblage and at the species levels, were similar but less significant when we considered only PAs of stricter management, as defined by IUCN categories I-IV (Dudley, 2008; see Supporting Information, Appendix S6). For shrub and herbaceous routes, the number of protected routes was even smaller than when all PAs were considered, leading to aberrant results.
Discussion
We compared the effect of PA coverage on bird species diversity, using assemblage indices (species richness, summed abundance) and individual species’ abundances.
At the assemblage level, we did not find significant differences in species richness or summed abundance between protected and unprotected sites, irrespective of whether vegetation structure was taken into account or not. In one sense, this is not surprising, particularly when it comes to species richness: according to the intermediate disturbance hypothesis, an area with low human-induced disturbance can have higher species richness than a pristine area (Roxburgh et al., 2004). Accordingly, Hiley et al., (2016) found lower alpha avian diversity in Mexican PAs than in unprotected areas. However, our results contrast with previous studies investigating this question such as Coetzee et al. (2014) or Gray et al. (2016), which found a positive effect of PAs on species richness and summed abundance, including in North America (Coetzee et al., 2014). These two studies being meta-analyses, it is possible that a publication bias against studies showing negative or null effects of PAs (discussed by Coetzee et al., 2014) artificially increased the difference they measured. This is even more so the case given that the underlying studies of the meta-analyses were often designed to measure the effect of anthropogenic pressures, using PAs as benchmarks, rather than measuring the effectiveness of PAs (e.g. Sinclair et al., 2002; Bihn et al., 2008; Wunderle et al., 2006, all used in Coetzee meta-analysis), and may thus have focused on particularly intact protected sites and/or in highly degraded non-protected sites. Conversely, our study may not be representative of studies at a global scale, for example if North American birds are less sensitive to human activities than other taxa in North America and/or birds in other regions, or if there is less contrast in human impacts in protected versus unprotected areas in North America than elsewhere. In addition, the lack of difference between protected and unprotected sites in terms of richness and abundance could also potentially be explained by a difference in species’ detectability (Boulinier et al., 1998) if PAs protect mainly species that are difficult to detect. This detection problem should not affect our result at the species level.
Even if overall species richness and abundance are similar, PAs may nonetheless have an effect on avian assemblages if different species respond differently to protection. We found that in routes whose vegetation is dominated by forests, PAs seem to have an overall positive effect on species’ abundance, but only for those species with forest as their main habitat. Forest PAs thus seem to maintain a more forest-typical bird assemblage than comparable unprotected forests. This effect was significant with the linear model, and with one (phyL), but not the other (phyBM) of the two phylogenetic linear models. This suggests that much of the effect attributed to habitat preferences under the linear model can actually be considered as phylogenetic difference, which is not surprising as bird habitat preferences and phylogeny are correlated. Phylogenetic models could theoretically allow us to measure the effectiveness of PAs in protecting species across phylogeny, and to check if some taxa were not effectively protected (e.g., they could highlight that a given family is not protected by PAs). However, to draw such conclusions, we would need to know how species are affected by PAs in each vegetation structure types, which is not the case here. Therefore, phylogenetic models give little information here, only highlighting that the difference in PAEFor between species habitat preferences is correlated with phylogeny. Moreover, all models indicate that species with low human-affinity (i.e., species that avoid human-impacted areas) are more favoured by forest PAs. This highlights that forest PAs protect species that are the most sensitive to human pressure, while species with high human-affinity, often benefiting from human presence, showed lower abundances inside PAs. Contrary to our expectation, and to previous results for common French birds (Devictor et al., 2007), we found no correlation between species’ population trends over the past 50 years and PAEFor. This may reflect the fact that our model included only relatively common species (i.e., observed on at least 100 routes in the studied years). It is thus possible that the most endangered species are favoured by PAs, but that we could not measure it.
Our models suggested that PAs in shrub areas have a beneficial impact on declining species and those with low human-affinity, whereas we did not find significant results for herbaceous areas. Given the scarcity of protected routes within both of these vegetation structure types, we do not consider these results robust or informative of the effectiveness of PAs, but they nonetheless emphasise the biases of BBS routes against shrub areas and herbaceous PAs (Appendix S4).
Given that PAs located in forests are not expected to favour the same species as PAs located in grasslands or shrub lands, we controlled for vegetation structure in our analyses of PA effects. However, this control masked the effect PAs may have had in preventing changes in vegetation structure (and associated changed in bird assemblages). For instance, given the vegetation structure categorisation we applied, the counterfactual for a protected forest was an unprotected forest, which does not take into account the possibility that the PA may have prevented the forest from being cleared. In other words, our approach does not measure the effect PAs can have on species diversity by preventing habitat destruction (that modifies vegetation structure type), only the effects PAs can have in preventing habitat degradation (not modifying the vegetation structure type), for example from natural forest to exploited forest, or from natural grassland to croplands.
Pairwise comparisons of protected versus unprotected sites, and thus the meta analyses from Geldmann et al. (2013), Coetzee et al. (2014) and Gray et al. (2016), can take into account the combined effects of habitat destruction and habitat degradation on species diversity, given that the counterfactual chosen may well have a different habitat structure than the protected site (e.g., a protected forest compared with an unprotected cropland). Nonetheless, defining the effectiveness measured in these meta-analyses is not straightforward, as it depends heavily on the choice of counterfactuals in underlying studies, which are defined directly by authors depending on their objectives. For instance, as discussed before, numerous studies used in the meta-analyses compare a highly degraded site with a protected site used as benchmark, in order to estimate the impact of anthropogenic degradation, which can lead to an overestimate of PA effectiveness. Other studies aimed to estimate PA effectiveness directly (e.g. Wasiolka and Blaum, 2011; Lee et al., 2007), but their choice of counterfactual was subjectively based on what authors considered likely to have happened to the protected site had it not been protected (Coetzee et al., 2014). Finally, some other studies used in meta-analyses were not particularly interested in differences between protected and unprotected sites, protection was only used as a covariate explaining potentially some noise around the signal the authors were interested in (e.g. Naidoo, 2004; McCarthy et al., 2010). Because of the diversity of approaches used in these meta-analyses, it is difficult to define precisely what has been measured as PA effectiveness. Although our approach does not allow us to measure the full effects of PAs, the difference we measured between protected and unprotected sites is defined statistically depending on the covariates included, which allows to understand clearly what is being included in measured effects of PA. A main advantage of large biodiversity monitoring datasets (such as breeding bird-monitoring schemes) in relation to pairwise comparisons is thus the possibility of applying a well-defined and repeatable control.
More broadly, our results highlight that clearly measuring PA effectiveness in conserving species diversity is impossible without defining precisely what is expected from them. In this study, we measured PAs effectiveness as the difference in abundance or richness between protected and unprotected sites. This definition assumes that PAs are expected to protect globally species diversity, and therefore gathers our ability to protect richest areas and to reduce human impacts on biodiversity in these areas. If PAs are expected to present higher diversity in terms of assemblage metrics (species richness or summed abundance), then we found no evidence in our analyses that PAs are effective. If PAs are expected to protect all species’ populations, then we did not find they were effective either, as for about half of the 149 species studied here we found a negative effect of PAs in forest. However, our results show that North-American forest PAs present higher abundances in forest species when compared with unprotected forest sites (especially for species with low affinity to human activities). That this result holds even though we found no significant difference in total abundance suggests that bird assemblages in protected forests are more forest-typical than those in unprotected forests. Our results thus indicate that forest PAs in North-American are contributing to prevent forest habitat degradation, and associated losses in the abundance of forest specialist species. BBS routes do not currently cover sufficiently well other habitats besides forest to allow us to investigate whether the same result applies to PAs with a different vegetation structure, but datasets with a bigger proportion of sampling points inside PAs, across all habitats, would help investigating this question.
Overall, our results emphasize the complexity of resolving a question that seemed so straightforward, and whose answer seemed so intuitive. In practice, understanding whether PAs are effective or not, and quantifying such effects, involves defining clearly what effect is being tested, on which facet of species diversity, and how to obtain appropriate counterfactuals.
Data accessibility statement
All data used in the study (birds and landscape covariates) are public and accessible to anybody. All sources are given with references.
Acknowledgments
We thank Jean-Yves Barnagaud for his insightful comments and suggestions concerning the analyses. We are grateful to all BBS birders and coordinators for the high quality data they have collected.