A comparison of two methods of calculating G sub(ST), a genetic measure of population differentiation

• Published in: American journal of botany Vol. 89; no. 3; pp. 460 - 465
• Main Authors: Culley, T M, Wallace, LE, Gengler-Nowak, K M, Crawford, D J
• Format: Journal Article
• Language: English
• Published: 01.03.2002
• ISSN: 0002-9122

G sub(ST) is a genetic statistic describing differentiation of populations and has frequently been compared with Hamrick and Godt's (1989) review of the plant literature. We show here that some comparisons may be inappropriate if G sub(ST) was calculated in a different way than that used by Hamrick and Godt (HG). An alternative method advocated by Nei is mathematically different from the HG technique, occasionally resulting in different G sub(ST) values. We reviewed 695 studies that appeared between 1990 and September 1999 that cited Hamrick and Godt (1989) and found that many of these calculated G sub(ST) according to Nei's method (46%), with the majority of these papers (61%) including comparisons to Hamrick and Godt's review. We suggest that if G sub(ST) estimates are compared across studies, it is most appropriate to calculate them the same way. However, we found that in most cases, the magnitude of difference in G sub(ST) values was small, suggesting that qualitative comparisons of G sub(ST) estimates between most studies are probably valid. Nevertheless, we have identified theoretical and empirical situations in which large differences in G sub(ST) values are likely to arise. Thus, we advise future investigators to carefully consider which method to use in calculating G sub(ST) for a given data set.