PT - JOURNAL ARTICLE AU - Yaron Granot AU - Saharon Rosset AU - Karl Skorecki TI - The Equidistance Index of Population Structure AID - 10.1101/033852 DP - 2015 Jan 01 TA - bioRxiv PG - 033852 4099 - http://biorxiv.org/content/early/2015/12/08/033852.short 4100 - http://biorxiv.org/content/early/2015/12/08/033852.full AB - Measures of population differentiation, such as FST, are traditionally derived from the ratio of genetic diversity within and between populations. However, the emergence of population clusters from multilocus analysis is a function of genetic structure (departures from panmixia) rather than diversity. If the populations are close to panmixia, slight differences between the mean pairwise distance within and between populations (low FST) can manifest as strong separation between the populations, thus population clusters are often evident even when the vast majority of diversity is partitioned within populations rather than between them. Moreover, because FST utilizes the mean rather than deviations from the mean, it does not directly reflect the strength of separation between population clusters. For any given FST value, clusters can be tighter (more panmictic) or looser (more stratified), and in this respect higher FST does not always imply stronger differentiation. In this study we propose substituting the mean in the FST equation with the standard deviation, thereby deriving a novel measure of population separability, denoted EST, which is more consistent with clustering and classification. To assess the utility of this metric, we ranked various human (HGDP) population pairs based on FST and EST and found substantial differences in ranking order. In some cases examined, most notably among isolated Amazonian tribes, EST ranking seems more consistent with demographic, phylogeographic and linguistic measures of classification compared to FST. Thus, EST may at times outperform FST in identifying evolutionarily significant differentiation.