Mixtures of Conditional Random Fields for Improved Structured Output Prediction

• Published in: IEEE transaction on neural networks and learning systems Vol. 28; no. 5; pp. 1233 - 1240
• Main Author: Kim, Minyoung
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Conditional random fields; Conditional random fields (CRFs); Data models; discriminative learning; Labeling; Learning algorithms; Machine learning; Mixture models; Mixtures; Periodic structures; Prediction algorithms; Predictions; Predictive models; Probabilistic models; Statistical analysis; Structured data; structured output prediction; Training
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2016.2521875

The conditional random field (CRF) is a successful probabilistic model for structured output prediction problems. In this brief, we consider to enlarge the representational capacity of CRF via mixture modeling. The motivation is that a single CRF can perform well if the data conform to the statistical dependence assumption imposed by the CRF model structure, whereas it may potentially fail to model the data that come from multiple different sources or domains. For the conventional conditional likelihood objective, we derive the expectation-maximization algorithm in conjunction with the direct gradient ascent method for learning a CRF mixture with sequence or image-structured data. In addition, we provide alternative mixture learning algorithms that aim to maximize either the classification margin or the sitewise conditional likelihood, which were previously shown to outperform the conventional estimator for single CRF models in a variety of situations. We demonstrate the improved prediction accuracy of the proposed mixture learning algorithms on several important sequence labeling problems.