Taxonomy-driven lumping for sequence mining

• Published in: Data mining and knowledge discovery Vol. 19; no. 2; pp. 227 - 244
• Main Authors: Bonchi, Francesco, Castillo, Carlos, Donato, Debora, Gionis, Aristides
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.10.2009; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Chemistry and Earth Sciences; Computer Science; Data mining; Data Mining and Knowledge Discovery; Datasets; Information Storage and Retrieval; Markov analysis; Physics; Software; Statistics for Engineering; Taxonomy; Query-log analysis; Markov models; Data mining; Sequence analysis; Spatial-data analysis
• ISSN: 1384-5810; 1573-756X
• DOI: 10.1007/s10618-009-0141-6

Given a taxonomy of events and a dataset of sequences of these events, we study the problem of finding efficient and effective ways to produce a compact representation of the sequences. We model sequences with Markov models whose states correspond to nodes in the provided taxonomy, and each state represents the events in the subtree under the corresponding node. By lumping observed events to states that correspond to internal nodes in the taxonomy, we allow more compact models that are easier to understand and visualize, at the expense of a decrease in the data likelihood. We formally define and characterize our problem, and we propose a scalable search method for finding a good trade-off between two conflicting goals: maximizing the data likelihood, and minimizing the model complexity. We implement these ideas in T axomo , a taxonomy-driven modeler, which we apply in two different domains, query-log mining and mining of moving-object trajectories. The empirical evaluation confirms the feasibility and usefulness of our approach.