Course material: The course is mainly based on
Presentation schedule:
Lecture  Topic  Material  Exercises  

1  Probability, entropy and inference  MK 2,3 
MK 2.4, 2.6 + continued, 2.7, 2.8 to be discussed in the class.
Exercises: MK 2.10, 2.14, 2.16ab, 2.18, 2.19, 2.26 MK 3.3, 3.4, 3.15 to be discussed in the class. Exercises: MK 3.1, 3.2, 3.5, 3.6, (3.7 Bonus), 3.8, 3.9  
2  Model comparison and Occam's raisor Graphical models 
MK 28 Bishop Chapter 8 [slides] Skip 8.1.3 
Exercises: MK 28.1, 28.2 only for model H_1, (28.3 Bonus) 8.1, 8.14 to be discussed in the class. Exercises: 8.3, 8.4, 8.7, 8.10, 8.11, 8.15 

3  Approximate inference 
cvm_sheets nips paper 
Run Belief Propagation for a
discrete optimization problem see: opgave Ising model. NB: Download LIBDAI from: LIBDAI. 

4 
Networks of binary neurons Markov processes Boltzmann Machines Mean field approximation Linear response approximation 
sheets Boltzmann Machines
reader chapter 3, 4 
reader
chapter 3 exercise 2 reader chapter 4 exercises 1, 2 

5  Monte Carlo Methods (2), HMC Bayesian inference with perceptron 
MK 29.129.5 MK 29.6,29.9 ,30.1, 30.3 MK 38, 39, 41 slides 
Exercise to compare MCMC with Belief Propagation
discrete optimization problem see: opgave Ising model. NB: Download LIBDAI from: LIBDAI. An example of Baysian inference in perceptron learning using MCMC methods. The files (Matlabfiles and instructions) needed to do this exercise can be found here: [mcmc_mackay.tar.gz]. 

6  Exercises and recap sofar  MK 2.10, MK 3.15 3.8 3.9 Example message passing in chain and loopy graph Example sequential dynamics Detailed balance, MF examples  
7  Discrete time control dynamic programming Bellman equation 
Bertsekas 25, 1314, 18, 2132 (2nd ed.) Bertsekas 25, 1012, 1627, 3032 (1nd ed.) Kappen ICML tutorial 1.2 slides up to 28 
Ex: Carry out the calculations needed to verify that J0(1)=2.7 and
J0(2)=2.818 in Bertsekas Example 3.2 on pg. 2325 in Copies 1b extra exercise 1, 2a,b  
8 
Continuous time control HamiltonJacobiBellman Equation Pontryagin Minimum Principle Stochastic differential equations Stochastic optimal control LQ examples, Portfolio management  Kappen ICML tutorial 1.3, 1.4 slides up to 69 
extra exercise 2a,b 

9 
Path integral control theory 
Kappen ICML tutorial 1.5, 1.6, 1.7 slides up to 93  extra exercise 2c, 3 
If time permits:
Lecture  Topic  Material  Exercises  

11  Path integral control theory MC Sampling solution Numerical examples (particle in a box, N joint arm, Robot learning) 
Kappen ICML tutorial 1.7 slides up to 127 
extra exercise 4,5
Matlab code for n joint problem Here is a directory of matlab files, which allows you to run and inspect the variational approximation for the n joint stochastic control problem as discussed in the tutorial text section 1.6.7. Type tar xvf njoints.tar to unpack the directory and simply run file1.m. In file1.m you can select demo1 (3 joint arm) or demo2 (10 joint arm). You can also try larger n but be sure to adjust eta for the smoothing of the variational fixed point equations. You can compare the results with exact cmputation (only recommendable for 2 joints) by setting METHOD='exact'. There is also an implementation of importance sampling (does not work very well) and Metropolis Hastings sampling (works nice, but not as stable as the variational approximation).  
12  Lasso  lasso slides 
Sparse regression computer exercise
 
13  Spike and slab Variational Garrote 
L0 slides George and McCulloch 1993 Kappen 2011 
 
6  Ising model  MK 31  MK 31.1, 31.3  
8a  Attractor neural networks  sheets attractors  
5  Perceptrons  DA 8.4 sheets supervised 1 sheets supervised 2 
DA 8.8, 8.9 