Personalized PCA: Decoupling Shared and Unique Features

In this paper, we tackle a significant challenge in PCA: heterogeneity. When data are collected from different sources with heterogeneous trends while still sharing some congruency, it is critical to extract shared knowledge while retaining the unique features of each source. To this end, we propose...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Shi, Naichen, Raed Al Kontar
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 08.02.2024
Subjects
Algorithms
Clustering
Constraints
Covariance matrix
Customization
Datasets
Decoupling
Feature extraction
Heterogeneity
Optimization
Orthogonality
Principal components analysis
Segmentation
Uniqueness
Online AccessGet full text

Cover

More Information
Summary:In this paper, we tackle a significant challenge in PCA: heterogeneity. When data are collected from different sources with heterogeneous trends while still sharing some congruency, it is critical to extract shared knowledge while retaining the unique features of each source. To this end, we propose personalized PCA (PerPCA), which uses mutually orthogonal global and local principal components to encode both unique and shared features. We show that, under mild conditions, both unique and shared features can be identified and recovered by a constrained optimization problem, even if the covariance matrices are immensely different. Also, we design a fully federated algorithm inspired by distributed Stiefel gradient descent to solve the problem. The algorithm introduces a new group of operations called generalized retractions to handle orthogonality constraints, and only requires global PCs to be shared across sources. We prove the linear convergence of the algorithm under suitable assumptions. Comprehensive numerical experiments highlight PerPCA's superior performance in feature extraction and prediction from heterogeneous datasets. As a systematic approach to decouple shared and unique features from heterogeneous datasets, PerPCA finds applications in several tasks, including video segmentation, topic extraction, and feature clustering.
ISSN:2331-8422