Analyzing inexact hypergradients for bilevel learning

• Published in: IMA journal of applied mathematics Vol. 89; no. 1; pp. 254 - 278
• Main Authors: Ehrhardt, Matthias J, Roberts, Lindon
• Format: Journal Article
• Language: English
• Published: Oxford University Press 21.06.2024
• Subjects: automatic differentiation; hyperparameter optimization; bilevel optimization

Estimating hyperparameters has been a long-standing problem in machine learning. We consider the case where the task at hand is modeled as the solution to an optimization problem. Here the exact gradient with respect to the hyperparameters cannot be feasibly computed and approximate strategies are r...