RT Journal Article
SR Electronic
T1 Emergent community agglomeration from data set geometry
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 109587
DO 10.1101/109587
A1 Chenchao Zhao
A1 Jun S. Song
YR 2017
UL http://biorxiv.org/content/early/2017/02/17/109587.abstract
AB In the statistical learning language, samples are snapshots of random vectors drawn from some unknown distribution. Such vectors usually reside in a high-dimensional Euclidean space, and thus, the “curse of dimensionality” often undermines the power of learning methods, including community detection and clustering algorithms, that rely on Euclidean geometry. This paper presents the idea of effective dissimilarity transformation (EDT) on empirical dissimilarity hyperspheres and studies its effects using synthetic and gene expression data sets. Iterating the EDT turns a static data distribution into a dynamical process purely driven by the empirical data set geometry and adaptively ameliorates the curse of dimensionality, partly through changing the topology of a Euclidean feature space ℝn into a compact hypersphere Sn. The EDT often improves the performance of hierarchical clustering via the automatic grouping information emerging from global interactions of data points. The EDT is not restricted to hierarchical clustering, and other learning methods based on pairwise dissimilarity should also benefit from the many desirable properties of EDT.PACS numbers: 89.20.Ff, 87.85.mg