Optimal clustering of frequency data with application to disease risk categorization
We provide a clustering procedure for a special type of dataset, known as frequency data, which counts the frequency of a certain binary outcome. An interpretation of the data as a discrete distribution enables us to extract statistical information, which we embed within an optimization-based framew...
Saved in:
Published in | IIE transactions Vol. 54; no. 8; pp. 728 - 740 |
---|---|
Main Authors | , |
Format | Journal Article |
Language | English |
Published |
Abingdon
Taylor & Francis
03.08.2022
Taylor & Francis Ltd |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | We provide a clustering procedure for a special type of dataset, known as frequency data, which counts the frequency of a certain binary outcome. An interpretation of the data as a discrete distribution enables us to extract statistical information, which we embed within an optimization-based framework. Our analysis of the resulting combinatorial optimization problem allows us to reformulate it as a more tractable network flow problem. This, in turn, enables the construction of exact algorithms that converge to the optimal solution in quadratic time. In addition, to be able to handle large-scale datasets, we provide two hierarchical heuristic algorithms that run in linearithmic time. Our moment-based method results in clustering solutions that are shown to perform well for a family of applications. We illustrate the benefits of our findings through a case study on HIV risk categorization within the context of large-scale screening through group testing. Our results on CDC data show that the proposed clustering framework consistently outperforms other popular clustering methods. |
---|---|
ISSN: | 2472-5854 2472-5862 |
DOI: | 10.1080/24725854.2021.1973158 |