PT - JOURNAL ARTICLE AU - Chenchao Zhao AU - Jun S. Song TI - Emergent community agglomeration from data set geometry AID - 10.1101/109587 DP - 2017 Jan 01 TA - bioRxiv PG - 109587 4099 - http://biorxiv.org/content/early/2017/02/17/109587.short 4100 - http://biorxiv.org/content/early/2017/02/17/109587.full AB - In the statistical learning language, samples are snapshots of random vectors drawn from some unknown distribution. Such vectors usually reside in a high-dimensional Euclidean space, and thus, the “curse of dimensionality” often undermines the power of learning methods, including community detection and clustering algorithms, that rely on Euclidean geometry. This paper presents the idea of effective dissimilarity transformation (EDT) on empirical dissimilarity hyperspheres and studies its effects using synthetic and gene expression data sets. Iterating the EDT turns a static data distribution into a dynamical process purely driven by the empirical data set geometry and adaptively ameliorates the curse of dimensionality, partly through changing the topology of a Euclidean feature space ℝn into a compact hypersphere Sn. The EDT often improves the performance of hierarchical clustering via the automatic grouping information emerging from global interactions of data points. The EDT is not restricted to hierarchical clustering, and other learning methods based on pairwise dissimilarity should also benefit from the many desirable properties of EDT.PACS numbers: 89.20.Ff, 87.85.mg