Learning to Detect an Odd Markov Arm

A multi-armed bandit with finitely many arms is studied when each arm is a homogeneous Markov process on an underlying finite state space. The transition law of one of the arms, referred to as the odd arm, is different from the common transition law of all other arms. A learner, who has no knowledge...

Full description

Saved in:
Bibliographic Details
Published in2019 IEEE International Symposium on Information Theory (ISIT) pp. 2554 - 2558
Main Authors Karthik, P. N., Sundaresan, Rajesh
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.07.2019
Subjects
Adaptive control
Aerospace electronics
History
Indexes
Markov processes
Upper bound
Online AccessGet full text

Cover

More Information
Summary:A multi-armed bandit with finitely many arms is studied when each arm is a homogeneous Markov process on an underlying finite state space. The transition law of one of the arms, referred to as the odd arm, is different from the common transition law of all other arms. A learner, who has no knowledge of the above transition laws, has to devise a sequential test to identify the index of the odd arm as quickly as possible, subject to an upper bound on the probability of error. For this problem, we derive an asymptotic lower bound on the expected stopping time of any sequential test of the learner, where the asymptotics is as the probability of error vanishes. Furthermore, we propose a sequential test, and show that the asymptotic behaviour of its expected stopping time comes arbitrarily close to that of the lower bound. Prior works deal with iid arms, whereas our work deals with Markov arms.
ISSN:2157-8117
DOI:10.1109/ISIT.2019.8849807