Abstract
We consider a sequential adaptive allocation problem which is formulated as a traditional two armed bandit problem but with one important modification: at each time step t, before selecting which arm to pull, the decision maker has access to a random variable χt which provides information on the reward in each arm. Performance is measured as the fraction of time an inferior arm (generating lower mean reward) is pulled. We derive a minimax lower bound that proves that in the absence of sufficient statistical "diversity" in the distribution of the covariate χ, a property that we shall refer to as lack of persistent excitation, no policy can improve on the best achievable performance in the traditional bandit problem without side information.
| Original language | English |
|---|---|
| Article number | 5714284 |
| Pages (from-to) | 1707-1713 |
| Number of pages | 7 |
| Journal | IEEE Transactions on Information Theory |
| Volume | 57 |
| Issue number | 3 |
| DOIs | |
| State | Published - Mar 2011 |
Bibliographical note
Funding Information:Manuscript received October 31, 2008; revised August 05, 2010; accepted August 05, 2010. Date of current version February 18, 2011. This work was supported in part by the NSF by Grant DMI-0447562 and in part by the US-Israel Binational Science Foundation (BSF) by Grant #2006075. A. Zeevi is with the Graduate School of Business, Columbia University, New York, NY 10027 USA (e-mail: [email protected]). A. Goldenshluger is with the Department of Statistics, Haifa University, Haifa 31905 Israel (e-mail: [email protected]). Communicated by A. Krzyzak, Associate Editor for Pattern Recognition, Statistical Learning, and Inference. Digital Object Identifier 10.1109/TIT.2011.2104450
Keywords
- Allocation rule
- inferior sampling rate
- lower bound
- side information
- two-armed bandit
ASJC Scopus subject areas
- Information Systems
- Computer Science Applications
- Library and Information Sciences