- One week given at Universidad Autonoma Madrid October 2010
- Form: Lectures, excercises, practicals
- For: Ma students and PhD students
- Lecturer: Bert Kappen

- ICML 2008 tutorial text will be published in a book Inference and Learning in Dynamical Models (Cambridge University Press 2010), edited by David Barber, Taylan Cemgil and Sylvia Chiappa. Here are the sheets of that tutorial.
- The tutorial website containts other tutorial material and pointers to useful further material.
- Dynamic programming and optimal control by Dimitri Bertsekas. Here are his slides for that course.

Date | Topic | Material | Recommended exercises | ||

1 | Feb 8 11-13 hours |
Discrete time control dynamic programming Bellman equation |
Bertsekas 2-5, 13-14, 18, 21-32 | Bertsekas 1.1 a and b, 1.2 | |

2 | Feb 9 11-13 hours |
Continuous time control Hamilton-Jacobi-Bellman Equation Pontryagin Minimum Principle Stochastic optimal control |
Kappen ICML tutorial 1.2, 1.3, 1.4 |
extra exercise 1, 2a,b Bertsekas 3.2 | |

3 | Feb 21 10-13 hours |
Dual control: the problem of joint inference and control Path integral control theory |
Kappen ICML tutorial 1.5,1.6, 1.7 | extra exercise 2c, 3 | |

4 | Feb 22 10-13 hours |
Stochastic optimal control Path integral control theory |
Kappen ICML tutorial 1.7 |
extra exercise 4,5
Matlab code for n joint problem Here is a directory of matlab files, which allows you to run and inspect the variational approximation for the n joint stochastic control problem as discussed in the tutorial text section 1.6.7. Type tar xvf njoints.tar to unpack the directory and simply run file1.m. In file1.m you can select demo1 (3 joint arm) or demo2 (10 joint arm). You can also try larger n but be sure to adjust eta for the smoothing of the variational fixed point equations. You can compare the results with exact cmputation (only recommendable for 2 joints) by setting METHOD='exact'. There is also an implementation of importance sampling (does not work very well) and Metropolis Hastings sampling (works nice, but not as stable as the variational approximation). |