A note on performance limitations in bandit problems with side information

Research output: Contribution to journalArticlepeer-review

Abstract

We consider a sequential adaptive allocation problem which is formulated as a traditional two armed bandit problem but with one important modification: at each time step t, before selecting which arm to pull, the decision maker has access to a random variable χt which provides information on the reward in each arm. Performance is measured as the fraction of time an inferior arm (generating lower mean reward) is pulled. We derive a minimax lower bound that proves that in the absence of sufficient statistical "diversity" in the distribution of the covariate χ, a property that we shall refer to as lack of persistent excitation, no policy can improve on the best achievable performance in the traditional bandit problem without side information.

Original languageEnglish
Article number5714284
Pages (from-to)1707-1713
Number of pages7
JournalIEEE Transactions on Information Theory
Volume57
Issue number3
DOIs
StatePublished - Mar 2011

Bibliographical note

Funding Information:
Manuscript received October 31, 2008; revised August 05, 2010; accepted August 05, 2010. Date of current version February 18, 2011. This work was supported in part by the NSF by Grant DMI-0447562 and in part by the US-Israel Binational Science Foundation (BSF) by Grant #2006075. A. Zeevi is with the Graduate School of Business, Columbia University, New York, NY 10027 USA (e-mail: [email protected]). A. Goldenshluger is with the Department of Statistics, Haifa University, Haifa 31905 Israel (e-mail: [email protected]). Communicated by A. Krzyzak, Associate Editor for Pattern Recognition, Statistical Learning, and Inference. Digital Object Identifier 10.1109/TIT.2011.2104450

Keywords

  • Allocation rule
  • inferior sampling rate
  • lower bound
  • side information
  • two-armed bandit

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this