Schedule:
Date  Topic  Chapter  Exercises  
1  April 2 18:0019:00 
Discrete time control dynamic programming Bellman equation 
Bertsekas 25, 1314, 18, 2132 (2nd ed.) Bertsekas 25, 1012, 1627, 3032 (1nd ed.) Kappen ICML tutorial 1.2 slides up to 34 
Ex: Verify that J0(1)=2.7 and J0(2)=2.818 in Bertsekas Example 3.2 on pg. 23 in Copies 1b  
2  April 2 19:0020:00 
Continuous time control HamiltonJacobiBellman Equation Pontryagin Minimum Principle Stochastic differential equations Stochastic optimal control LQ examples, Portfolio management 
Kappen ICML tutorial 1.3,1.4 slides up to 59 
extra exercise 2a,b 

3  April 4 16:0017:00 
Path integral control theory 
Kappen ICML tutorial 1.5, 1.6, 1.7 slides up to 93  extra exercise 2c, 3  
4  April 4 17:0018:00 
Path integral control theory MC Sampling solution Numerical examples (particle in a box, N joint arm, Robot learning) 
Kappen ICML tutorial 1.7 slides up to 127 
extra exercise 4,5 Matlab code for n joint problem See below. 