Special Topic Course "Theory of Reinforcement Learning"
In connection with our reading groups we offer the possibility for Honors and undergraduate students to take a Special Topic Course worth 6 units based on the book being read. The reading group might start before you start your course but this is not a big problem, though we encourage participation even before formally starting. In semester 2 of 2011 we are offering "Theory of Reinforcement Learning" as well as an alternative overlapping variant that include graphical models. If you are interested in taking a special topic course along these lines please contact Peter Sunehag or Stephen Gould (graphical models) ua.ude.una|emantsal.emantsrif#ua.ude.una|emantsal.emantsrif .
Assessment
The exact details of assessment are decided in a individual study contract which will be based on
- Being an active participant in the reading group which includes presenting some sections
- Implemented case study
- The quality of a written report
- Assignments (for the graphical models version)
Theory of Reinforcement Learning
Reinforcement Learning (RL) is an approach of Artificial Intelligence based on learning to solve a task through trial and error. Its an area which have had many practical successes ( http://umichrl.pbworks.com/w/page/7597597/Successes-of-Reinforcement-Learning) in for example game play (chess, backgammon, …) and robotics (autonomous helicopter flight) and most (if not all) of AI can be described as RL problems
From 1.June'11 the reinforcement learning reading group (http://grla.wikidot.com/frl ) will read "Neuro-dynamic programming" by Dimitri P. Bertsekas and John Tsitsiklis. This book present the modern theory of reinforcement learning. A more practical companion book is "Reinforcement Learning: an introduction" by Sutton and Barto http://www.incompleteideas.net/sutton/book/the-book.html . The most important topic of the course is reinforcement learning with function approximation which, unlike tabular methods, allows for generalization across a state space as well as the ability to deal with continuous problems.
Graphical Models (and reinforcement learning)
This course will cover both reinforcement learning and graphical models.
The reinforcement learning component will be a subset of the "Theory of
Reinforcement Learning" course described above.
Graphical Models provide a probabilistic modelling language for
representing and reasoning about complex domains under uncertainty. In
this course we will cover various graphical models representations,
including Bayesian Networks and Markov random fields, and inference and
learning algorithms. For this component, material will be covered in a
series of lectures and additional readings. The course will involve both
theory and programming assignments.
Learning Outcomes:
- The student will be able to explain: Reinforcement Learning (RL) problems; Markov Decision Processes (MDPs) (finite, infinite, continuous); The algorithms presented in the book (see table of contents http://www.athenasc.com/ndpcontents.html); The proof techniques presented like stochastic approximation theory (Chapter 4)
- The student will be able to state what guarantees exist for the algorithms presented in the book.
- Experience of implementing reinforcement learning with function approximation
Schedule
We meet on Wednesdays 11:30-12:30 in a meeting room at level 2 RSISE. Note that the main content of the course, reinforcement learning with function approximation is contained in the massive chapter 6 which will be studied in September and October.
- June Chapter 1-2
- July Chapter 3-4
- August Chapter 4-5
- Sep-Oct Chapter 6
- Nov Chapter 7-8
The graphical models lectures will be given by Steve Gould in September. The exact time and location will be decided with the participants when the course has started.
Contact
Peter Sunehag <ua.ude.una|gahenus.retep#ua.ude.una|gahenus.retep>
Phone: 02-61257709
Room: RSISE B225
