RT Journal Article
SR Electronic
T1 Critical Assessment of Metagenome Interpretation – a benchmark of computational metagenomics software
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 099127
DO 10.1101/099127
A1 Alexander Sczyrba
A1 Peter Hofmann
A1 Peter Belmann
A1 David Koslicki
A1 Stefan Janssen
A1 Johannes Dröge
A1 Ivan Gregor
A1 Stephan Majda
A1 Jessika Fiedler
A1 Eik Dahms
A1 Andreas Bremges
A1 Adrian Fritz
A1 Ruben Garrido-Oter
A1 Tue Sparholt Jørgensen
A1 Nicole Shapiro
A1 Philip D. Blood
A1 Alexey Gurevich
A1 Yang Bai
A1 Dmitrij Turaev
A1 Matthew Z. DeMaere
A1 Rayan Chikhi
A1 Niranjan Nagarajan
A1 Christopher Quince
A1 Lars Hestbjerg Hansen
A1 Søren J. Sørensen
A1 Burton K. H. Chia
A1 Bertrand Denis
A1 Jeff L. Froula
A1 Zhong Wang
A1 Robert Egan
A1 Dongwan Don Kang
A1 Jeffrey J. Cook
A1 Charles Deltel
A1 Michael Beckstette
A1 Claire Lemaitre
A1 Pierre Peterlongo
A1 Guillaume Rizk
A1 Dominique Lavenier
A1 Yu-Wei Wu
A1 Steven W. Singer
A1 Chirag Jain
A1 Marc Strous
A1 Heiner Klingenberg
A1 Peter Meinicke
A1 Michael Barton
A1 Thomas Lingner
A1 Hsin-Hung Lin
A1 Yu-Chieh Liao
A1 Genivaldo Gueiros Z. Silva
A1 Daniel A. Cuevas
A1 Robert A. Edwards
A1 Surya Saha
A1 Vitor C. Piro
A1 Bernhard Y. Renard
A1 Mihai Pop
A1 Hans-Peter Klenk
A1 Markus Göker
A1 Nikos Kyrpides
A1 Tanja Woyke
A1 Julia A. Vorholt
A1 Paul Schulze-Lefert
A1 Edward M. Rubin
A1 Aaron E. Darling
A1 Thomas Rattei
A1 Alice C. McHardy
YR 2017
UL http://biorxiv.org/content/early/2017/01/09/099127.abstract
AB In metagenome analysis, computational methods for assembly, taxonomic profiling and binning are key components facilitating downstream biological data interpretation. However, a lack of consensus about benchmarking datasets and evaluation metrics complicates proper performance assessment. The Critical Assessment of Metagenome Interpretation (CAMI) challenge has engaged the global developer community to benchmark their programs on datasets of unprecedented complexity and realism. Benchmark metagenomes were generated from newly sequenced ~700 microorganisms and ~600 novel viruses and plasmids, including genomes with varying degrees of relatedness to each other and to publicly available ones and representing common experimental setups. Across all datasets, assembly and genome binning programs performed well for species represented by individual genomes, while performance was substantially affected by the presence of related strains. Taxonomic profiling and binning programs were proficient at high taxonomic ranks, with a notable performance decrease below the family level. Parameter settings substantially impacted performances, underscoring the importance of program reproducibility. While highlighting current challenges in computational metagenomics, the CAMI results provide a roadmap for software selection to answer specific research questions.