Neural collapse with unconstrained features

Neural collapse is an emergent phenomenon in deep learning that was recently discovered by Papyan, Han and Donoho. We propose a simple unconstrained features model in which neural collapse also emerges empirically. By studying this model, we provide some explanation for the emergence of neural colla...

Full description

Saved in:
Bibliographic Details
Published inSampling theory, signal processing, and data analysis Vol. 20; no. 2
Main Authors Mixon, Dustin G., Parshall, Hans, Pi, Jianzong
Format Journal Article
LanguageEnglish
Published Cham Springer International Publishing 01.11.2022
Subjects
Abstract Harmonic Analysis
Machine Learning
Mathematics
Mathematics and Statistics
Original Article
Signal,Image and Speech Processing
68T07
Neural collapse
Unconstrained features
Neural networks
42C15
Equiangular tight frames
Online AccessGet full text

Cover

More Information
Summary:Neural collapse is an emergent phenomenon in deep learning that was recently discovered by Papyan, Han and Donoho. We propose a simple unconstrained features model in which neural collapse also emerges empirically. By studying this model, we provide some explanation for the emergence of neural collapse in terms of the landscape of empirical risk.
ISSN:2730-5716
2730-5724
DOI:10.1007/s43670-022-00027-5