Penalized Linear Regression

• Published in: Machine Learning with Spark and Python pp. 1 - 2
• Main Author: Bowles, Michael
• Format: Book Chapter
• Language: English
• Published: United States John Wiley & Sons 2020; John Wiley & Sons, Incorporated; John Wiley & Sons, Inc
• Edition: 2nd Edition
• Subjects: binary classification problems; General Engineering & Project Administration; General References; multiclass classification problems; ordinary least squares; penalized linear regression; Python code implementation; regression algorithm
• ISBN: 1119561930; 9781119561934
• DOI: 10.1002/9781119562023.ch4

This chapter discusses the performance of penalized linear models. It develops an extended family of methods for controlling the overfitting inherent in ordinary least squares. The chapter also explains how the penalty method determines the nature of the solution and the type of information that is available about the solution. It describes principles of operation for two modern algorithms for solving the penalized regression minimization problem and Python code implementing the main features of the algorithms in order to have a concrete instantiation of the core of the algorithms to make the principals of operation clear. The plain regression problem (numeric features and numeric targets) served as the exemplar for in‐depth coverage of algorithms. The chapter shows several extensions to broaden the use cases to include binary classification problems, multiclass classification problems, problems with nonlinear relationship between attributes and outcomes, and problems with non‐numeric features.