Course Schedule

Weekday Regular Schedule

Group	Type	Hours	Location
01	Lecture	Monday 13-16	Orentstein 103

Resources

Introduction to Multi-Armed Bandits, Aleksandrs Slivkins

Bandit Algorithms Tor Lattimore and Csaba Szepesvari

Detailed Schedule

Week	Date	Lecture topics	references	Lecture Slides	Lecture Scribes
1	Jan 1	Introduction and Stochastic Bandits	Chapters 1 and Introduction from Slivkins, Chapter 5.3 from Lattimore and Szepesvari, Sleeping bandits from Regret bounds for sleeping experts and bandits by Robert Kleinberg, Alexandru Niculescu-Mizil, Yogeshwer Sharma	Hebrew Notes 1	Scribe 1
2	Jan 8	Lower Bounds MAB	Chapters 2 from Slivkins, Sleeping bandits from Regret bounds for sleeping experts and bandits by Robert Kleinberg, Alexandru Niculescu-Mizil, Yogeshwer Sharma	Hebrew Notes 2	Scribe 2
3	Jan 15	Bayesian bandits and Thompson sampling algorithm	Chapters 3 from Slivkins, Near-optimal Regret Bounds for Thompson Sampling by SHIPRA AGRAWAL and NAVIN GOYAL,	Hebrew Notes 3	Scribe 3
week 4	Jan 22	No new material
4	Jan 29	Lipschitz bandits	Chapter 4 from Slivkins, The Value of Knowing a Demand Curve: Bounds on Regret for On-line Posted-Price Auctions by Robert Kleinberg and Tom Leighton	Hebrew Notes 4	Scribe 4
5	Feb 5	Adversarial costs: Full feedback	Chapter 5 from Slivkins, Learning, Regret minimization, and Equilibria (Section 4.3) by A. Blum and Y. Mansour From External to internal regret (Section 7) by A. Blum and Y. Mansour	Hebrew Notes 5	Scribe 5
6	Feb 12	Adversarial costs: MAB	Reduction: previous years class notes THE NONSTOCHASTIC MULTIARMED BANDIT PROBLEM by P. AUER, N. CESA-BIANCHI, Y. FREUND, and R. SCHAPIRE Explore no more: Improved high-probability regret bounds for non-stochastic bandits by G. Neu	Hebrew Notes 6	Scribe 6
7	Feb 19	Linear Cost	Full information: Efficient algorithms for online decision problems Kalai and Vempala 2005 Reduction and Brycentic spanner Online linear optimization and adaptive routing Awerbuch and Kleinberg 2008 Stochastic: Chapter 19 Bandit Algorithms book (Lattimore and Szepesvari) Additional resources: Slivkins book Chapter 7 Stochastic Linear Optimization under Bandit Feedback Dani, Hayes, Kakade 2008	Hebrew Notes 7	Scribe 7
8	Feb 26	Contextual Bandits	Lecture notes from 2018 course Efficient Algorithms for Adversarial Contextual Learning Vasilis Syrgkanis, Akshay Krishnamurthy, Robert E. Schapire, ICML 2016	Hebrew Notes 8	Scribe 8
9	March 4	Zero-sum games and applications	Lecture notes from 2018/19 and 2009/10 course Slivkins chapter 9	Hebrew Notes 9
10	March 11	Swap and and correlated equilibrium	Lecture notes from 2018/19	Hebrew Notes 10