Singular-Value-Decomposition-Based Matrix Surgery

This paper is motivated by the need to stabilise the impact of deep learning (DL) training for medical image analysis on the conditioning of convolution filters in relation to model overfitting and robustness. We present a simple strategy to reduce square matrix condition numbers and investigate its...

Full description

Saved in:
Bibliographic Details
Published inEntropy (Basel, Switzerland) Vol. 26; no. 8; p. 701
Main Authors Ghafuri, Jehan, Jassim, Sabah
Format Journal Article
LanguageEnglish
Published Switzerland MDPI AG 01.08.2024
MDPI
Subjects
Algorithms
condition number
Conditioning
Convolution
Datasets
Deep learning
Image analysis
Image filters
Impact analysis
Investigations
Medical imaging
Medical imaging equipment
Neural networks
Regularization
Singular value decomposition
Spatial distribution
Surgery
SVD surgery
topological data analysis
United States > US
SVD surgery
singular value decomposition
topological data analysis
condition number
Online AccessGet full text

Cover

More Information
Summary:This paper is motivated by the need to stabilise the impact of deep learning (DL) training for medical image analysis on the conditioning of convolution filters in relation to model overfitting and robustness. We present a simple strategy to reduce square matrix condition numbers and investigate its effect on the spatial distributions of point clouds of well- and ill-conditioned matrices. For a square matrix, the SVD surgery strategy works by: (1) computing its singular value decomposition (SVD), (2) changing a few of the smaller singular values relative to the largest one, and (3) reconstructing the matrix by reverse SVD. Applying SVD surgery on CNN convolution filters during training acts as spectral regularisation of the DL model without requiring the learning of extra parameters. The fact that the further away a matrix is from the non-invertible matrices, the higher its condition number is suggests that the spatial distributions of square matrices and those of their inverses are correlated to their condition number distributions. We shall examine this assertion empirically by showing that applying various versions of SVD surgery on point clouds of matrices leads to bringing their persistent diagrams (PDs) closer to the matrices of the point clouds of their inverses.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:1099-4300
1099-4300
DOI:10.3390/e26080701