Efficient Clustering for Stretched Mixtures: Landscape and Optimality

This paper considers a canonical clustering problem where one receives unlabeled samples drawn from a balanced mixture of two elliptical distributions and aims for a classifier to estimate the labels. Many popular methods including PCA and k-means require individual components of the mixture to be s...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Wang, Kaizheng, Yan, Yuling, Díaz, Mateo
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 27.11.2021
Subjects
Affine transformations
Clustering
First order algorithms
Optimization
Statistical methods
Online AccessGet full text

Cover

More Information
Summary:This paper considers a canonical clustering problem where one receives unlabeled samples drawn from a balanced mixture of two elliptical distributions and aims for a classifier to estimate the labels. Many popular methods including PCA and k-means require individual components of the mixture to be somewhat spherical, and perform poorly when they are stretched. To overcome this issue, we propose a non-convex program seeking for an affine transform to turn the data into a one-dimensional point cloud concentrating around \(-1\) and \(1\), after which clustering becomes easy. Our theoretical contributions are two-fold: (1) we show that the non-convex loss function exhibits desirable geometric properties when the sample size exceeds some constant multiple of the dimension, and (2) we leverage this to prove that an efficient first-order algorithm achieves near-optimal statistical precision without good initialization. We also propose a general methodology for clustering with flexible choices of feature transforms and loss objectives.
ISSN:2331-8422