Dynamic miss-counting algorithms: finding implication and similarity rules with confidence pruning

Dynamic miss-counting (DMC) algorithms are proposed which find all implication and similarity rules with confidence pruning but without support pruning. To handle data sets with a large number of columns, we propose dynamic pruning techniques that can be applied during data scanning. DMC counts the...

Full description

Saved in:
Bibliographic Details
Published inProceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073) pp. 501 - 511
Main Authors Fujiwara, S., Ullman, J.D., Motwani, R.
Format Conference Proceeding
LanguageEnglish
Published IEEE 2000
Subjects
Online AccessGet full text

Cover

Loading…
More Information
Summary:Dynamic miss-counting (DMC) algorithms are proposed which find all implication and similarity rules with confidence pruning but without support pruning. To handle data sets with a large number of columns, we propose dynamic pruning techniques that can be applied during data scanning. DMC counts the numbers of rows in which each pair of columns disagree instead of counting the number of hits. DMC deletes a candidate as soon as the number of misses exceeds the maximum number of misses allowed for that pair. We also propose several optimization techniques that reduce the required memory size significantly. We evaluated our algorithms by using four data sets, viz. Web access logs, a Web page-link graph, news documents and a dictionary. These data sets have between 74,000 and 700,000 items. Experiments show that DMC can find high-confidence rules for such large data sets efficiently.
ISBN:9780769505060
0769505066
ISSN:1063-6382
2375-026X
DOI:10.1109/ICDE.2000.839449