Repeatability of the searching process in reviews of restoration outcomes

• Published in: Restoration ecology Vol. 29; no. 8
• Main Authors: Romanelli, João Paulo, Silva, Luiz G. M., Gonçalves, Maria Carolina P., Naves, Rafaela P., Almeida, Danilo Roberti Alves, Resende, Angélica Faria, Rodrigues, Ricardo Ribeiro
• Format: Journal Article
• Language: English
• Published: Malden, USA Wiley Periodicals, Inc 01.11.2021; Blackwell Publishing Ltd
• Subjects: ecological restoration; Ecosystem restoration; Environmental restoration; evidence synthesis; evidence‐based restoration; International organizations; Literature reviews; meta-analysis; Methods; reliability; Reproducibility; restoration reviews; review methodology; sampling; Searching; Statistical analysis; systematic review; United Nations
• ISSN: 1061-2971; 1526-100X
• DOI: 10.1111/rec.13496

In the recently declared United Nations Decade on Ecosystem Restoration (2021–2030), evidence‐based research has an essential role in summarizing restoration outcomes toward broad generalizations to advance restoration practice and science globally. However, to present trustworthy, high‐quality recommendations, evidence reviews should be based on rigorous methods that minimize bias and enhance systematicity, transparency, objectivity, comprehensiveness, and repeatability. To assess the current value of restoration reviews in terms of methodological repeatability at the searching stage, a fundamental review stage, we evaluated a sample of meta‐analyses and narrative syntheses (n = 79) and critically appraised how searches were conducted based on the information reported within each study. By assessing whether review methods were reported in sufficient detail to be repeated by an independent party, we found that repeatability varied substantially by the standard we used. Overall, reviews performed relatively poorly, with a median score of 3 out of 9 points available, and a mean score of 3.6. Most reviews (n = 76/79) failed in effectively reporting all necessary information to allow repeatability at this stage. We found no statistically significant differences considering review types, suggesting that the type of synthesis alone does not solely reflect the repeatability of a review. Toward increasingly repeatable, reliable reviews in restoration ecology, we recommend that authors incorporate and apply principles of systematic reviews and maps in their review process, and suggest that environmental journals should broadly emphasize ways of performing rigorous reviews, as well as increase efforts for the publication of open access review protocols.