Can Entropic Regularization Be Replaced by Squared Euclidean Distance Plus Additional Linear Constraints

There are two main families of on-line algorithms depending on whether a relative entropy or a squared Euclidean distance is used as a regularizer. The difference between the two families can be dramatic. The question is whether one can always achieve comparable performance by replacing the relative...

Full description

Saved in:
Bibliographic Details
Published inLearning Theory pp. 653 - 654
Main Author Warmuth, Manfred K.
Format Book Chapter Conference Proceeding
LanguageEnglish
Published Berlin, Heidelberg Springer Berlin Heidelberg 2006
Springer
SeriesLecture Notes in Computer Science
Subjects
Applied sciences
Artificial intelligence
Computer science; control theory; systems
Euclidean Distance
Exact sciences and technology
Neural Information Processing System
Perceptron Algorithm
Relative Entropy
Weight Vector
Disjunction
Entropy
Probability distribution
Divergence
Euclidean theory
Artificial intelligence
Online AccessGet full text

Cover

More Information
Summary:There are two main families of on-line algorithms depending on whether a relative entropy or a squared Euclidean distance is used as a regularizer. The difference between the two families can be dramatic. The question is whether one can always achieve comparable performance by replacing the relative entropy regularization by the squared Euclidean distance plus additional linear constraints. We formulate a simple open problem along these lines for the case of learning disjunctions.
ISBN:3540352945
9783540352945
ISSN:0302-9743
1611-3349
DOI:10.1007/11776420_48