Four proofs of Gittins’ multiarmed bandit theorem

Esther Frostig; Gideon Weiss

doi:10.1007/s10479-013-1523-0

Four proofs of Gittins’ multiarmed bandit theorem

Esther Frostig, Gideon Weiss

Department of Statistics

Research output: Contribution to journal › Article › peer-review

Abstract

We study four proofs that the Gittins index priority rule is optimal for alternative bandit processes. These include Gittins’ original exchange argument, Weber’s prevailing charge argument, Whittle’s Lagrangian dual approach, and Bertsimas and Niño-Mora’s proof based on the achievable region approach and generalized conservation laws. We extend the achievable region proof to infinite countable state spaces, by using infinite dimensional linear programming theory.

Original language	English
Pages (from-to)	127-165
Number of pages	39
Journal	Annals of Operations Research
Volume	241
Issue number	1-2
DOIs	https://doi.org/10.1007/s10479-013-1523-0
State	Published - 1 Jun 2016

Bibliographical note

Funding Information:
G. Weiss’s research supported in part by Israel Science Foundation Grants 249/02, 454/05, 711/09 and 286/13.

Funding Information:
E. Frostig’s research supported in part by Network of Excellence Euro-NGI.

Publisher Copyright:
© 2014, Springer Science+Business Media New York.

Keywords

Bandit problems
Dynamic programming
Gittins index
Linear programming

ASJC Scopus subject areas

Decision Sciences (all)
Management Science and Operations Research

Access to Document

10.1007/s10479-013-1523-0

Cite this

@article{4be326cfa62f4d96852e1044ba786c66,

title = "Four proofs of Gittins{\textquoteright} multiarmed bandit theorem",

abstract = "We study four proofs that the Gittins index priority rule is optimal for alternative bandit processes. These include Gittins{\textquoteright} original exchange argument, Weber{\textquoteright}s prevailing charge argument, Whittle{\textquoteright}s Lagrangian dual approach, and Bertsimas and Ni{\~n}o-Mora{\textquoteright}s proof based on the achievable region approach and generalized conservation laws. We extend the achievable region proof to infinite countable state spaces, by using infinite dimensional linear programming theory.",

keywords = "Bandit problems, Dynamic programming, Gittins index, Linear programming",

author = "Esther Frostig and Gideon Weiss",

note = "Funding Information: G. Weiss{\textquoteright}s research supported in part by Israel Science Foundation Grants 249/02, 454/05, 711/09 and 286/13. Funding Information: E. Frostig{\textquoteright}s research supported in part by Network of Excellence Euro-NGI. Publisher Copyright: {\textcopyright} 2014, Springer Science+Business Media New York.",

year = "2016",

month = jun,

day = "1",

doi = "10.1007/s10479-013-1523-0",

language = "English",

volume = "241",

pages = "127--165",

journal = "Annals of Operations Research",

issn = "0254-5330",

publisher = "Springer Netherlands",

number = "1-2",

}

TY - JOUR

T1 - Four proofs of Gittins’ multiarmed bandit theorem

AU - Frostig, Esther

AU - Weiss, Gideon

N1 - Funding Information: G. Weiss’s research supported in part by Israel Science Foundation Grants 249/02, 454/05, 711/09 and 286/13. Funding Information: E. Frostig’s research supported in part by Network of Excellence Euro-NGI. Publisher Copyright: © 2014, Springer Science+Business Media New York.

PY - 2016/6/1

Y1 - 2016/6/1

N2 - We study four proofs that the Gittins index priority rule is optimal for alternative bandit processes. These include Gittins’ original exchange argument, Weber’s prevailing charge argument, Whittle’s Lagrangian dual approach, and Bertsimas and Niño-Mora’s proof based on the achievable region approach and generalized conservation laws. We extend the achievable region proof to infinite countable state spaces, by using infinite dimensional linear programming theory.

AB - We study four proofs that the Gittins index priority rule is optimal for alternative bandit processes. These include Gittins’ original exchange argument, Weber’s prevailing charge argument, Whittle’s Lagrangian dual approach, and Bertsimas and Niño-Mora’s proof based on the achievable region approach and generalized conservation laws. We extend the achievable region proof to infinite countable state spaces, by using infinite dimensional linear programming theory.

KW - Bandit problems

KW - Dynamic programming

KW - Gittins index

KW - Linear programming

UR - http://www.scopus.com/inward/record.url?scp=84891524486&partnerID=8YFLogxK

U2 - 10.1007/s10479-013-1523-0

DO - 10.1007/s10479-013-1523-0

M3 - Article

AN - SCOPUS:84891524486

SN - 0254-5330

VL - 241

SP - 127

EP - 165

JO - Annals of Operations Research

JF - Annals of Operations Research

IS - 1-2

ER -

Four proofs of Gittins’ multiarmed bandit theorem

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this