Abstract
Motivation Despite the recent progress in genome sequencing and assembly, many of the currently available assembled genomes come in a draft form. Such draft genomes consist of a large number of genomic fragments (scaffolds), whose positions and orientations along the genome are unknown. While there exists a number of methods for reconstruction of the genome from its scaffolds, utilizing various computational and wet-lab techniques, they often can produce only partial error-prone scaffold assemblies. It therefore becomes important to compare and merge scaffold assemblies produced by different methods, thus combining their advantages and highlighting present conflicts for further investigation. These tasks may be labor intensive if performed manually.
Results We present CAMSA—a tool for comparative analysis and merging of two or more given scaffold assemblies. The tool (i) creates an extensive report with several comparative quality metrics; (ii) constructs the most confident merged scaffold assembly; and (iii) provides an interactive framework for a visual comparative analysis of the given assemblies. Among the CAMSA features, only scaffold merging can be evaluated in comparison to existing methods. Namely, it resembles the functionality of assembly reconciliation tools, although their primary targets are somewhat different. Our evaluations show that CAMSA produces merged assemblies of comparable or better quality than existing assembly reconciliation tools while being the fastest in terms of the total running time.
Availability CAMSA is distributed under the MIT license and is available at http://cblab.org/camsa/.
1. Introduction
While genome sequencing technologies are constantly evolving, researchers are still unable to read complete genomic sequences at once from organisms of interest. So, genome reading is usually done in multiple steps, which involve both in vitro and in silico methods. It starts with reading small genomic fragments, called reads, originating from unknown locations in the genome. Modern shotgun sequencing technologies can easily produce millions of reads. The problem then becomes to assemble them into the complete genome. Existing de novo genome assembly algorithms can usually assemble reads into longer genomic fragments, called contigs, that are typically interweaved in the genome with highly polymorphic and/or repetitive regions. The next step is to construct scaffolds, i.e., sequences of (oriented) contigs along the genome interspaced with gaps. The last but not least step is genome finishing that recovers genomic sequences inside the gaps within the scaffolds.
Unfortunately, the quality of scaffolds (e.g., exposing severe fragmentation) for many genomes makes the finishing step infeasible. As a result, the majority of currently available genomes come in a draft form represented by a large number of scaffolds rather than complete chromosomes [35]. This emphasizes the need for improving the assembly quality of genomes by constructing longer scaffolds from the given ones,1 which we refer to as the scaffold assembly problem. In other words, the scaffold assembly problem asks for reconstruction of the order of input scaffolds along the genome chromosomes.
A number methods have been recently proposed to address the scaffold assembly problem by utilizing various types of additional information and/or in vitro experiments. These methods are based on jumping libraries [20, 40, 22, 17, 27, 11, 7], long error-prone reads (such PacBio or MinION reads) [45, 5, 6, 8, 24], homology relationship between multiple genomes [3, 1, 2], wet-lab experiments such as the fluorescence in situ hybridization (FISH) [36, 41], genome maps [32, 42, 28], higher order chromatin interactions [9], and so on. Depending on the nature and accuracy of utilized information and techniques, assemblies produces by different methods may still be incomplete and contain errors, thus deviating from each other. Moreover, some scaffolding methods (e.g., based on FISH or HiC data) can produce assemblies, where the (strand-based) orientation of some assembled scaffolds is yet to be determined.
It therefore becomes crucial to determine what parts of different assemblies are consistent with and/or complement each other, and what parts are conflicting with other assemblies (or even within the same assembly). Furthermore, some scaffold assemblies may utilize only a fraction of the input scaffolds (e.g., homology-based assembly methods do not take into account unannotated scaffolds), thus posing a problem of analyzing and comparing assemblies of varying subsets of scaffolds. Comparative analysis of scaffold assemblies produced by different methods can help the researchers to combine their advantages, and highlight potential conflicts for further investigation. These tasks may be labor-intensive if performed manually.
While there exists a number of methods [47, 48, 34, 43, 30, 46, 44] for reconciling multiple assemblies of the same organism, they all are limited only to oriented scaffolds and thus are in-applicable to scaffold assemblies that include unoriented scaffolds. Furthermore, some of these methods require a reference genome sequence, which is often unavailable for non-model organisms. On the other hand, reconciliation methods that operate in de-novo fashion often process the input assemblies progressively, which makes such methods sensitive to the order of the input assemblies and affects the quality of the reconciled assembly.
We present CAMSA, a tool for comparative analysis and de-novo merging of scaffold assemblies. CAMSA takes as an input two or more assemblies of the same set of scaffolds and generates a comprehensive comparative report for them. Input assemblies can include both oriented and unoriented scaffolds, which enables CAMSA to process assemblies from the full range of scaffolding techniques (both in silico and in vitro). The generated comparative report not only contains multiple numerical characteristics for the input assemblies, but also provides an interactive framework, allowing one to visually analyze and compare the input scaffold assemblies at regions of interest. CAMSA also computes a merged assembly, combining the input assemblies into a more comprehensive one that resolves conflicts and determines orientation of unoriented scaffolds in the most confident way. The non-progressive nature of merging in CAMSA eliminates the dependency on the order of input scaffold assemblies. We remark that CAMSA can be utilized at different stages of the genome assembly process and be applied to assemblies of various genomic fragments, ranging from contigs to superscaffolds. In particular, CAMSA input can include results of other assembly reconciliation methods.
2. Methods
Assembly Analysis and Visualization
For the purpose of comparative analysis and visualization of the input scaffold assemblies, CAMSA utilizes the breakpoint graphs, the data structure traditionally used for analysis of gene orders across multiple species [4]. We will refer to the breakpoint graph constructed on a set of scaffold assemblies as the scaffold assembly graph (SAG).
We start with the case of assemblies with no unoriented scaffolds. Then each assembly A can be viewed as a set of sequences of oriented scaffolds. We represent A as an individual scaffold assembly graph SAG(A) with two types of edges: directed edges (scaffold edges) encoding scaffolds in A, and undirected edges (assembly edges) representing scaffold adjacencies and connecting extremities (tails/heads) of the corresponding scaffold edges (Fig. 1A,B,C).
We find it convenient to refer to each assembly edge as an assembly point. Equivalently, an assembly point in A can be represented by an ordered pair of oriented scaffolds. We specify the orientation of a scaffold s, either by a sign (+s or −s) or by an overhead arrow . For example, and represent the same assembly point between scaffolds s1 and s2following each other head-to-head. Clearly, any assembly is completely defined by the set of its assembly points.
To construct the scaffold assembly graph SAG(A1,…, Ak) of multiple input assemblies A1,…, Ak, we represent them as individual graphs SAG(A1),…, SAG(Ak), where the undirected edges in each SAG(Ai) are colored into unique color. Then the graph SAG(A1,…, Ak) can be viewed as the superposition of individual graphs SAG(A1),…, SAG(Ak) and can be obtained by gluing the identically labeled directed edges. So the graph SAG(A1,…, Ak) consists of (directed, labeled) scaffold edges encoding scaffolds and (undirected, unlabeled) assembly edges of k colors encoding assembly points in different input assemblies (Fig. 1D). We will refer to edges of color Ai (i.e., coming from SAG(Ai)) as Ai-edges. The assembly edges connecting the same two vertices x and y form the multiedge {x, y} in SAG(A1,…, Ak). The multicolor of {x, y} is defined as the union of the colors of individual edges connecting x and y.
We define the (ordinary) degree odeg(x) of a vertex x in SAG(A1,…, Ak) as the number of assembly edges incident to x. We distinguish it from the multidegree mdeg(x) defined as the number of adjacent vertices that are connected to x with assembly edges.
When all assemblies A1,…, Ak agree on a particular assembly point {x, y}, the graph SAG(A1,…, Ak) contains a multi-edge {x, y} composed of edges of all k different colors. In other words, both vertices x and y in this case have degree k and multidegree 1. For a vertex z in SAG(A1,…, Ak), odeg(z) ≠ k or mdeg(z) ≠ 1 indicate some type of inconsistency between the assemblies.
We classify an individual assembly points {x, y} as follows. Let S be the multicolor of the multiedge {x, y} in SAG(A1,…, Ak).
unique if |S| = 1, i.e., the assembly point {x, y} is present only in a single assembly;
in-conflicting within assembly A ∈ S if x or y is incident any other A-edges besides {x, y} (e.g., Fig. 2C);
out-conflicting if there exist two distinct assemblies A and B such that A contains {x, y} (i.e., A ∈ S), and B contains {x, z} with z ≠ y or {y, z} with z ≠ x (e.g., Fig. 2A).
Dealing with Unoriented scaffolds
While conventional multiple breakpoint graphs are constructed for sequences of oriented genes, in CAMSA we extend scaffold assembly graphs to support assemblies that may include oriented as well as unoriented scaffolds.
In addition to (oriented) assembly points formed by pairs of oriented scaffolds, we now consider semi-oriented and unoriented assembly points.
A semi-oriented assembly point represents an adjacency between an oriented scaffold and an unoriented one. For example, and denote the same semi-oriented assembly point, where scaffold s1 is oriented and s2 is not (as emphasized by a missing overhead arrow). Similarly, an unoriented assembly point represents an adjacency between two unoriented scaffolds. For example, (s1, s2) and (s2, s1) denote the same unoriented assembly point between unoriented scaffolds s1 and s2.
We define a realization of an assembly point p as any oriented assembly point that can be obtained from p by orienting unoriented scaffolds. We denote the set of orientations of p as R(p). If p is oriented, then it has a single realization equal to p itself (i.e., R(p) = {p}); if p is semi-oriented, then it has two realizations (i.e., |R(p)| = 2); and if p is unoriented, then it has four realizations (i.e., |R(p)| = 4). For example,
In the scaffold assembly graph, we add assembly edges encoding all realizations of semi-/un-oriented assembly points and refer to such edges as candidate, in contrast to actual assembly edges encoding oriented assembly points.
We extend the in-/out-conflicting classification to semi-oriented and unoriented assembly points as follows. An assembly point is in-/out-conflicting if all its realizations are such, except that we do not consider two realizations as in-conflicting to each other. Similarly, an assembly point is in-/out-semiconflicting if some but not all of its realizations are in-/out-conflicting (e.g., Fig. 2B,D illustrate pairs of out-semiconflicting and in-semiconflicting assembly points, respectively).
Merging Assemblies
CAMSA can resolve conflicts in the input assemblies by merging them into a single (not self-confliciting) merged assembly that is most consistent with the input ones. The merged assembly is also used to determine orientation of (some) unoriented scaffolds in one input assemblies that is most confident and/or consistent with other input assemblies. In other words, the merged assembly helps to identify realizations of (some) semi-/un-oriented assembly points that are most consistent with other assemblies. Namely, for each semi-/un-oriented assembly point, the merged assembly contains either only one or none of its realizations; and in the former case, the included realization defines the most confident orientation of the corresponding unoriented scaffolds.
Assembly merging performed by CAMSA is based on how often each assembly point appears in the input assemblies as well as on the (optional) confidence of each such appearance. Namely, for each assembly point p in an input assembly A, CAMSA allows to specify the confidence weight cwA(p) from the interval [0, 1], which is then assigned to the corresponding assembly edge(s) (Fig. 3A). The confidence weights are expected to reflect the confidence level of the assembly methods in what they report as scaffold adjacencies (e.g., heuristic methods should probably have smaller confidence as compared to more reliable wet-lab techniques). By default, all actual assembly edges have the confidence weight equal 1, and all candidate assembly edges have weight 0.75 (these default values can be overwritten by the user).
For any oriented assembly B (viewed as a set of oriented assembly points), we define the consistency score csB (A) of an input assembly A with respect to B as , where
We pose the assembly merging problem (AMP) as follows.
Problem 1 (Assembly Merging Problem, AMP). Given assemblies A1,…, Ak of the same set of scaffolds S, find an assembly M of S containing only oriented assembly points such that
M is not self-conflicting (i.e., does not contain any in-conflicting assembly points);
csM (Ai) is maximized;
for every assembly point p ∈ A1 ∪ · · · ∪ Ak, at most one of its realizations is present in M (i.e., |M ∩ R(p)| ≤ 1).
For a solution M to the AMP, the condition (i) implies that the assembly edges in SAG(M) form a matching. Furthermore, M is assumed to correspond to the genome, which may be subject to additional constraints such as having all chromosomes linear (e.g., for vertebrate genomes) or having a single chromosome (e.g., for bacterial genomes). These constraints are translated for M as the absence in SAG(M) of cycles formed by alternating assembly and scaffold edges (for a unichromosomal circular genome, such a cycle can be present in SAG(M) only if it includes all scaffold edges).
To address the AMP, we start with construction of the (weighted) merged scaffold assembly graph MSAG(A1,…, Ak) from SAG(A1,…, Ak) by replacing each assembly multi-edge with an ordinary assembly edge of the weight equal the total weight of the corresponding multi-edge (Fig. 3). So, MSAG(A1,…, Ak) is the graph with two types of edges: unweighted directed scaffolds edges and weighted undirected assembly edges. The AMP is then can be reformulated as the following restricted maximum matching problem (RMMP) on the graph G = MSAG(A1,…, Ak):
Problem 2 (Restricted Maximum Matching Problem, RMMP). Given a merged scaffold assembly graph G, find a subset M of assembly edges in G such that
M is a matching;
M has maximum weight;
there are no cycles in SAG(M).
Let M be a solution to the RMMP. Then the graph SAG(M) consists of scaffold edges forming a perfect matching and assembly edges from M forming a (possibly non-perfect) matching by the condition (i). Thus SAG(M) is formed by collection of paths and cycles, whose edges alternate between scaffold and assembly edges. Furthermore, by the condition (iii), SAG(M) consists entirely of alternating paths. A similar optimization problem, where the number of paths and the number cycles in the resulting SAG(M) are fixed, is known to be NP-complete [10], leaving a little hope for the RMMP to have a polynomial-time solution. Instead, CAMSA employs two merging heuristic solutions building upon the previously proposed algorithms [31, 10] as described below in this section.
Greedy merging heuristics. For a given merged scaffold assembly graph G, this strategy starts with the graph H consisting of scaffold edges from G and then iteratively enriches H with assembly edges so that no cycles are created in H. At any stage of this process, H is considered as a collection of alternating paths, some of which are merged into a longer path by adding a corresponding assembly edge. The paths to merge are selected based on the confidence weight of their linking assembly edge. The final graph H constructed this way defines M as the set of assembly edges in H (and so SAG(M) = H).
Maximum matching heuristics. For a given merged scaffold assembly graph G, this first computes the maximum weighted matching M ′ formed by assembly edges of G. Namely, CAMSA employs the NetworkX library [19] implementation of the blossom algorithm [14] for computing M ′.2 For the maximum weighted matching M ′, CAMSA looks for cycles in SAG(M ′) (notice that all cycles in SAG(M ′) are vertex-disjoint) and removes an assembly edge of the lowest confidence weight from each such cycle. These edges are also removed from M ′ to form M so that SAG(M) consists entirely of alternating paths.
We remark that before solving the RMMP for G = MSAG(A1,…, Ak), CAMSA allows to remove assembly edges from G that have weight smaller than the weight threshold specified by the user (by default, this threshold is set to 0, i.e., no edges are removed). The removal of small-weighted assembly edges may be desirable if one wants to restrict attention only to assembly points of certain confidence level (e.g., assembly points coming either from individual highly-reliable assemblies or as a consensus from multiple assemblies). When such removal of low-confidence edges is performed, it is important to do so before (not after) solving the RMMP, since otherwise these edges may introduce a bias for an inclusion of high-confidence edges into the merged assembly M.
3. Structure of CAMSA Report
The results of comparative analysis and assembly merging performed by CAMSA are presented to the user in the form of an interactive report. The report is generated in a form of a JavaScript-powered HTML file, readily accessible for viewing/working in any modern Internet browser (for locally generated reports, Internet connection is not required). Many of the report sections are also available in the form of text files, making them accessible for machine processing. All tables in the report are powered by the DataTables JavaScript library [21], which provides flexible and dynamic filtering, sorting, and searching capabilities.
The first section of the CAMSA report presents aggregated characteristics of each input assemblies as compared to the others:
the number of oriented, semi-oriented, and unoriented assembly points;
the number of in-/out-conflicting assembly points;
the number of in-/out-semiconflicting assembly points;
the number of nonconflicting assembly points;
the number of assembly points that participate in the merged assembly.
The second section of the CAMSA report focuses on consistency across various subsets of input assemblies. For each subset, it gives characteristics similar to the ones in the first section, but the values here are aggregated over all assemblies in the subset. The subsets are listed as a bar diagram in the descending order of the number of unique assembly points they contain (Fig. 4). Such statistics eliminates the need of running CAMSA separately on any assemblies subsets and allows the user to easily identify groups of assemblies that agree/conict among themselves the most. We remark that each assembly point is counted only once: for the set of assemblies that contains this assembly point (but not for any its smaller subset). Since the the number of all subsets of input assemblies can be large, CAMSA allows the user to specify the number of top subsets to be displayed.
The third section of the CAMSA report provides statistics for each assembly point within each assembly. Extensive interactive filtering allows the user to select assembly points of interest, as well as to export the filtered results, creating problem-/ region-/ fragment-focused analysis pipelines. We remark that statistical characteristics (e.g., whether an assembly point is in-/out-conflicting or in-/out-semiconflicting) are computed with respect to all of the input assemblies.
The fourth section of the CAMSA report provides statistics for each assembly point aggregated over all of the input assemblies (Fig. 5). In contrast to the third section, each assembly point is shown here exactly once, and the sources column shows the set of assemblies where this assembly point is present (e.g., in Fig. 5 the assembly point is present in A1, A2, and A3). Again, CAMSA provides extensive filtering to enable a focused analysis of assembly points of interest. The result of assembly points filtration can further be exported in the same format, which is utilized for CAMSA input files (i.e., list of assembly points in a tab-separated format).
Besides the text-based representation and export, the CAMSA report also provides an interactive visualization and further graphical export of assembly points in the form of the scaffold assembly graph. A vector-based interactive graph visualization is created using the Cytoscape.js library [13]. This visualization has a dynamic graph layout and supports filtration of graph components. We allow the user to choose from several Cytoscape.js graph layouts; the default layout comes from [12].
At any point the current image of the scaffold assembly graph can be exported from the report into a PNG file.
The time required for graph visualization heavily depends on the chosen layout and the underlying graph complexity. In cases when visualization inside the report takes too much time, we provide the following workarounds. The assembly points can be exported in a text file and then converted into a DOT file describing the corresponding scaffold assembly graph, whose visualization then can be constructed with GraphViz [15]. Alternatively, one can choose to export the SAG subgraph induced by the filtered assembly points into a JSON file, which can further be processed with the desktop Cytoscape software [38].
4. Evaluation
While merging of multiple input scaffold assemblies is just one of the features of the CAMSA framework, it is the only one that resembles existing tools, namely those performing assembly reconciliation. We therefore feel obliged to compare its performance to such tools, even though we pose CAMSA as a meta-tool that can take as an input the results of various scaffolding methods, including assembly reconciliation tools.
We evaluated the assembly merging in CAMSA by running it on multiple scaffold assemblies of genomes of different sizes from the GAGE project [37]. While CAMSA can be used at any stage of genome scaffolding, in this evaluation we applied it to the results of initial scaffolding of contigs based on jumping libraries. We chose the following four scaffolders for performing such task: ScaffMatch [29], SOAPdenovo2 [27], SGA [39], and SSPACE [7]. The input to these scaffolders was formed by contigs and jumping libraries assembled and corrected by Allpaths-LG [16], which are provided by GAGE. The scaffold assemblies produced by these scaffolders were used as an input to CAMSA as well as to Metassembler [46] and GAM-NGS [44] assembly reconciliation tools.3 To demonstrate the advantages of CAMSA as a meta-tool, we also run it on the four aforementioned scaffold assemblies combined with the two reconciled assemblies produced by Metassembler and GAM-NGS, and denoted as CAMSA (+GM) in the evaluation results.
All tools were run on the same computer system with dual Intel Xeon E5-2670 2.6GHz 8-core processors and 64GB of RAM. First, we measured the running time of each tool. Then we assessed the quality of the resulting scaffold assemblies (formed by merged scaffolds) with the number of metrics computed by QUAST [18] with–scaffolds flag. Below we present most important metrics, while the complete QUAST reports for both input (Tables S8, S9, S10) and resulting scaffold assemblies (Tables S11, S12, S13) are provided in Supplement A. Namely, we mostly concern the following QUAST metrics:
# contigs: in our evaluation, the contigs counted by QUAST correspond the merged scaffolds; so their number measures the contiguity of the resulting scaffold assemblies.
# misassemblies (miss.): number of breakpoints in the merged scaffolds, for which the left and right flanking sequences align in the reference genome to different strands / chromosomes (inversions / translocations), or on the same stand and chromosome with a gap of ≥ 1000bp between each other (relocations).
# local misassemblies (local miss.): number of relocations with a gap in the range from 85bp to 1000bp.
NA50: the maximum length L such that the fragments of length ≥ L obtained from the merged scaffolds by breaking them at misassembly sites cover at least 50% of assembly. NA75: similar to NA50, but with 75% coverage of the assembly.
Table 1 demonstrates that CAMSA is the fastest among the tools in comparison. We separately benchmarked the data preparation and processing. We remark that depending on the format of input scaffold assemblies as well as the overall assembly pipeline, the data preparation step may be not required or take significantly different time. For CAMSA in this evaluation data preparation involves the conversion of scaffold assemblies from FASTA format into the set of assembly points,4 using a utility script based on NUCmer software [23]. For GAM-NGS, one needs to align the jumping libraries onto the input scaffold assemblies as well as onto the intermediate reconciled assemblies (progressively generated from the input assemblies). The former alignments were treated as data preparation (since they may be readily available from the assembly pipeline), while the latter alignments are generally unavailable and thus were treated as data processing. For Metassembler, no data preparation was required since all alignments are performed internally.
Table 2 shows the quality of the scaffold assemblies produced by different tools. In all datasets, the assembly produced by CAMSA was either the best or very close to the best in each of the metrics. We remark that in some cases CAMSA (+GM) takes advantage of the reconciled assemblies and demonstrates better results than CAMSA. In other cases, however, having the reconciled assemblies turns out to be disadvantageous due to the elevated presence of misassemblies in them. This emphasizes the fact that assembly reconciliation/merging is sensitive to the quality of input assemblies and should be interpreted with caution. The comparative report in CAMSA can greatly help in identification of conflicting assembly points (indicating potential misassemblies), enabling their targeted analysis.
5. Discussion
CAMSA addresses the current deficiency of tools for automated comparison and analysis of multiple assemblies of the same set scaffolds. Since there exist numerous methods and techniques for scaffold assembly, identifying similarities and dissimilarities across assemblies produced by different methods is beneficial both for the developers of scaffold assembly algorithms and for the researchers focused on improving draft assemblies of specific organisms.
We remark that CAMSA expects as an input a list of assembly points, which differs from the output produced by some conventional scaffolding tools. This inspired us to develop a set of utility scripts that automate the input/output conversion process for CAMSA (e.g., from/to formats like FASTA, AGPv2, or GRIMM), and include them in the CAMSA distribution.
We further plan to enrich the graph-based analysis in CAMSA with various pattern matching techniques, enabling a better classification of assembly conflicts based on their origin (e.g., conflicting scaffold orders, wrong orientation of scaffolds, or different resolution of assemblies). We also plan on adding a reference mode, so that classification of assembly points in the input assemblies can be done with respect to a known reference genome, rather than just with respect to each other.
We also remark that CAMSA is currently utilized in the study of Anopheles mosquito genomes [33], where multiple research laboratories (including ours) work on improving the existing assemblies for a set of mosquito species.
Footnotes
*The work is supported by the National Science Foundation under the grant No. IIS-1462107.
1 We remark that contigs can be viewed as scaffolds with no gaps. So, under scaffolds we understand both contigs and scaffolds.
2 The blossom algorithm computes a maximal weighted matching in a graph in O(V3) time, where V is the number of vertices.
3 We also considered GARM [30], but were unable to run it on any GAGE dataset, facing issues similar to those reported in [46].
4 We remark that conversion, for example, from NCBI AGPv2 format (rather than FASTA) would be much faster.