A Mathematical View of Weights-of-Evidence, Conditional Independence, and Logistic Regression in Terms of Markov Random Fields

• Published in: Mathematical geosciences Vol. 46; no. 6; pp. 691 - 709
• Main Author: Schaeben, Helmut
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.08.2014; Springer Nature B.V
• Subjects: Approximation; Chemistry and Earth Sciences; Computer Science; Earth and Environmental Science; Earth Sciences; Geology; Geotechnical Engineering & Applied Earth Sciences; Hierarchies; Hydrogeology; Indicators; Logistics; Markov analysis; Markov models; Mathematical models; Mathematics; Physics; Regression; Rendering; Statistics for Engineering; Graphical models; Weights-of-evidence; Conditional independence; Logistic regression model; Markov random field; Log-linear models
• DOI: 10.1007/s11004-013-9513-y
• ISSN: 1874-8961

New light is shed on mathematical methods of potential modeling from the point of view of Markov random fields. In particular, weights-of-evidence and logistic regression models are discussed in terms of graphical models possessing Markov properties, where the notion of conditional independence is essential, and will be related to log-linear models. While weights-of-evidence with respect to indicator predictor variables and logistic regression with unrestricted predictor variables model conditional probabilities of an indicator random target variable, the subject of log-linear models is the joint probability of random variables. The relationship to log-linear models leads to a likelihood ratio test of conditional independence, rendering an omnibus test of conditional independence restricted by a normality assumption obsolete. Moreover, it reveals a hierarchy of methods comprising weights-of-evidence, logistic regression without interaction terms, and logistic regression including interaction terms, where each former method is a special case of the consecutive latter method. The assumptions of conditional independence of all predictor variables given the target variable lead to logistic regression without interaction terms. Violations of conditional independence are compensated exactly by corresponding interaction terms, no cumbersome approximate corrections are needed. Thus, including interaction terms into logistic regression models is an appropriate means to account for lacking conditional independence. Logistic regression exempts from the burden to worry about lack of conditional independence. Eventually, the relationship to log-linear models renders logistic regression with indicator predictor variables optimum for discrete predictor variables. Weights-of-evidence applies for indicator predictor variables only, logistic regression applies without restrictions of the type of predictor variables and approximates the proper distribution in the general case.