Minimization Techniques for Piecewise Differentiable Functions: The l1Solution to an Overdetermined Linear System

• Published in: SIAM journal on numerical analysis Vol. 15; no. 2; pp. 224 - 241
• Main Authors: Bartels, Richard H., Conn, Andrew R., Sinclair, James W.
• Format: Journal Article
• Language: English
• Published: Society for Industrial and Applied Mathematics 01.04.1978
• Subjects: Algorithms; Factorization; Linear equations; Linear programming; Linear systems; Mathematical functions; Mathematical vectors; Mathematics; Matrices; Simplex method
• ISSN: 0036-1429; 1095-7170

A new algorithm is presented for computing a vector x which satisfies a given m by$n (m > n \geqq 2)$linear system in the sense that the l1norm is minimized. That is, if A is a matrix having m columns a1, ⋯, ameach of length n, and b is a vector with components β1, ..., βm, then x is selected so that φ(x) = | ATx- b|1= ∑m i = 1| aT ix- βi| is as small as possible. Such solutions are of interest for the "robust" fitting of a linear model to data. The function φ is directly minimized in a finite number of steps using techniques borrowed from Conn's approach toward minimizing piecewise differentiable functions. In these techniques if x is any point and ALstands for the submatrix consisting of those columns ajfrom A for which the corresponding residuals aT jx- βjare zero, then the discontinuities in the gradient of φ at x are handled by making use of the projector onto the null space of AT L. Attention has been paid both to numerical stability and efficiency in maintaining and updating a factorization of ALfrom which the necessary projector is obtainable. The algorithm compares favorably with the best so far reported for the linear l1problem, and it can easily be extended to handle linear constraints.