Optimal clustering of frequency data with application to disease risk categorization

We provide a clustering procedure for a special type of dataset, known as frequency data, which counts the frequency of a certain binary outcome. An interpretation of the data as a discrete distribution enables us to extract statistical information, which we embed within an optimization-based framew...

Full description

Saved in:
Bibliographic Details
Published inIIE transactions Vol. 54; no. 8; pp. 728 - 740
Main Authors Aprahamian, Hrayer, El-Amine, Hadi
Format Journal Article
LanguageEnglish
Published Abingdon Taylor & Francis 03.08.2022
Taylor & Francis Ltd
Subjects
Online AccessGet full text

Cover

Loading…
More Information
Summary:We provide a clustering procedure for a special type of dataset, known as frequency data, which counts the frequency of a certain binary outcome. An interpretation of the data as a discrete distribution enables us to extract statistical information, which we embed within an optimization-based framework. Our analysis of the resulting combinatorial optimization problem allows us to reformulate it as a more tractable network flow problem. This, in turn, enables the construction of exact algorithms that converge to the optimal solution in quadratic time. In addition, to be able to handle large-scale datasets, we provide two hierarchical heuristic algorithms that run in linearithmic time. Our moment-based method results in clustering solutions that are shown to perform well for a family of applications. We illustrate the benefits of our findings through a case study on HIV risk categorization within the context of large-scale screening through group testing. Our results on CDC data show that the proposed clustering framework consistently outperforms other popular clustering methods.
ISSN:2472-5854
2472-5862
DOI:10.1080/24725854.2021.1973158